LLMResult(generations=[[Generation(text='\n\nQ: What did the fish say when he hit the wall? \nA: Dam.', generation_info={'finish_reason': 'stop', 'logprobs': None})], [Generation(text='\n\nThe Moon \n\nThe moon is high in the midnight sky,\nSparkling like a star above.\nThe night so peaceful, so serene,\nFilling up the air with love.\n\nEver changing and renewing,\nA never-ending light of grace.\nThe moon remains a constant view,\nA reminder of life’s gentle pace.\n\nThrough time and space it guides us on,\nA never-fading beacon of hope.\nThe moon shines down on us all,\nAs it continues to rise and elope.', generation_info={'finish_reason': 'stop', 'logprobs': None})], [Generation(text='\n\nQ. What did one magnet say to the other magnet?\nA. "I find you very attractive!"', generation_info={'finish_reason': 'stop', 'logprobs': None})], [Generation(text="\n\nThe world is charged with the grandeur of God.\nIt will flame out, like shining from shook foil;\nIt gathers to a greatness, like the ooze of oil\nCrushed. Why do men then now not reck his rod?\n\nGenerations have trod, have trod, have trod;\nAnd all is seared with trade; bleared, smeared with toil;\nAnd wears man's smudge and shares man's smell: the soil\nIs bare now, nor can foot feel, being shod.\n\nAnd for all this, nature is never spent;\nThere lives the dearest freshness deep down things;\nAnd though the last lights off the black West went\nOh, morning, at the brown brink eastward, springs —\n\nBecause the Holy Ghost over the bent\nWorld broods with warm breast and with ah! bright wings.\n\n~Gerard Manley Hopkins", generation_info={'finish_reason': 'stop', 'logprobs': None})], [Generation(text='\n\nQ: What did one ocean say to the other ocean?\nA: Nothing, they just waved.', generation_info={'finish_reason': 'stop', 'logprobs': None})], [Generation(text="\n\nA poem for


'stop', 'logprobs': None})], [Generation(text="\n\nA poem for you\n\nOn a field of green\n\nThe sky so blue\n\nA gentle breeze, the sun above\n\nA beautiful world, for us to love\n\nLife is a journey, full of surprise\n\nFull of joy and full of surprise\n\nBe brave and take small steps\n\nThe future will be revealed with depth\n\nIn the morning, when dawn arrives\n\nA fresh start, no reason to hide\n\nSomewhere down the road, there's a heart that beats\n\nBelieve in yourself, you'll always succeed.", generation_info={'finish_reason': 'stop', 'logprobs': None})]], llm_output={'token_usage': {'completion_tokens': 504, 'total_tokens': 528, 'prompt_tokens': 24}, 'model_name': 'text-davinci-003'})


Scenario 2: Tracking an LLM in a chain
Then we can create a chain using a prompt template, and then track the initial prompt and the final response in Argilla.
from langchain.callbacks import ArgillaCallbackHandler, StdOutCallbackHandler
from langchain.llms import OpenAI
from langchain.chains import LLMChain
from langchain.prompts import PromptTemplate


argilla_callback = ArgillaCallbackHandler(
dataset_name="langchain-dataset",
api_url=os.environ["ARGILLA_API_URL"],
api_key=os.environ["ARGILLA_API_KEY"],
)
callbacks = [StdOutCallbackHandler(), argilla_callback]
llm = OpenAI(temperature=0.9, callbacks=callbacks)

template = """You are a playwright. Given the title of play, it is your job to write a synopsis for that title.
Title: {title}
Playwright: This is a synopsis for the above play:"""
prompt_template = PromptTemplate(input_variables=["title"], template=template)
synopsis_chain = LLMChain(llm=llm, prompt=prompt_template, callbacks=callbacks)

test_prompts = [{"title": "Documentary about Bigfoot in Paris"}]
synopsis_chain.apply(test_prompts)

> Entering new LLMChain chain...
Prompt after formatting:
You are a playwright. Given the title of play, it is your job to write a synopsis for that title.
Title: Documentary about Bigfoot in Paris
Playwright: This is a synopsis for the above play:

> Finished chain.


> Finished chain.

[{'text': "\n\nDocumentary about Bigfoot in Paris focuses on the story of a documentary filmmaker and their search for evidence of the legendary Bigfoot creature in the city of Paris. The play follows the filmmaker as they explore the city, meeting people from all walks of life who have had encounters with the mysterious creature. Through their conversations, the filmmaker unravels the story of Bigfoot and finds out the truth about the creature's presence in Paris. As the story progresses, the filmmaker learns more and more about the mysterious creature, as well as the different perspectives of the people living in the city, and what they think of the creature. In the end, the filmmaker's findings lead them to some surprising and heartwarming conclusions about the creature's existence and the importance it holds in the lives of the people in Paris."}]
Scenario 3: Using an Agent with Tools
Finally, as a more advanced workflow, you can create an agent that uses some tools. So that ArgillaCallbackHandler will keep track of the input and the output, but not about the intermediate steps/thoughts, so that given a prompt we log the original prompt and the final response to that given prompt.
Note that for this scenario we'll be using Google Search API (Serp API) so you will need to both install google-search-results as pip install google-search-results, and to set the Serp API Key as os.environ["SERPAPI_API_KEY"] = "..." (you can find it at https://serpapi.com/dashboard), otherwise the example below won't work.
from langchain.agents import AgentType, initialize_agent, load_tools
from langchain.callbacks import ArgillaCallbackHandler, StdOutCallbackHandler
from langchain.llms import OpenAI

argilla_callback = ArgillaCallbackHandler(
dataset_name="langchain-dataset",
api_url=os.environ["ARGILLA_API_URL"],
api_key=os.environ["ARGILLA_API_KEY"],
)
callbacks = [StdOutCallbackHandler(), argilla_callback]
llm = OpenAI(temperature=0.9, callbacks=callbacks)

tools = load_tools(["serpapi"], llm=llm, callbacks=callbacks)
agent = initialize_agent(
tools,
llm,
agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION,
callbacks=callbacks,
)
agent.run("Who was the first president of the United States of America?")


> Entering new AgentExecutor chain...
I need to answer a historical question
Action: Search
Action Input: "who was the first president of the United States of America"
Observation: George Washington
Thought: George Washington was the first president
Final Answer: George Washington was the first president of the United States of America.

> Finished chain.

'George Washington was the first president of the United States of America.'


Context
Context provides user analytics for LLM powered products and features.
With Context, you can start understanding your users and improving their experiences in less than 30 minutes.
In this guide we will show you how to integrate with Context.
Installation and Setup
$ pip install context-python --upgrade
Getting API Credentials
To get your Context API token:
Go to the settings page within your Context account (https://with.context.ai/settings).
Generate a new API Token.
Store this token somewhere secure.
Setup Context
To use the ContextCallbackHandler, import the handler from Langchain and instantiate it with your Context API token.
Ensure you have installed the context-python package before using the handler.
import os

from langchain.callbacks import ContextCallbackHandler

token = os.environ["CONTEXT_API_TOKEN"]

context_callback = ContextCallbackHandler(token)
Usage
Using the Context callback within a chat model
The Context callback handler can be used to directly record transcripts between users and AI assistants.
Example
import os

from langchain.chat_models import ChatOpenAI
from langchain.schema import (
SystemMessage,
HumanMessage,
)
from langchain.callbacks import ContextCallbackHandler

token = os.environ["CONTEXT_API_TOKEN"]

chat = ChatOpenAI(
headers={"user_id": "123"}, temperature=0, callbacks=[ContextCallbackHandler(token)]
)

messages = [
SystemMessage(
content="You are a helpful assistant that translates English to French."
),
HumanMessage(content="I love programming."),
]

print(chat(messages))
Using the Context callback within Chains
The Context callback handler can also be used to record the inputs and outputs of chains. Note that intermediate steps of the chain are not recorded - only the starting inputs and final outputs.
Note: Ensure that you pass the same context object to the chat model and the chain.
Wrong:
chat = ChatOpenAI(temperature=0.9, callbacks=[ContextCallbackHandler(token)])
chain = LLMChain(llm=chat, prompt=chat_prompt_template, callbacks=[ContextCallbackHandler(token)])
Correct:
handler = ContextCallbackHandler(token)
chat = ChatOpenAI(temperature=0.9, callbacks=[callback])
chain = LLMChain(llm=chat, prompt=chat_prompt_template, callbacks=[callback])
Example
import os


from langchain.chat_models import ChatOpenAI
from langchain import LLMChain
from langchain.prompts import PromptTemplate
from langchain.prompts.chat import (
ChatPromptTemplate,
HumanMessagePromptTemplate,
)
from langchain.callbacks import ContextCallbackHandler

token = os.environ["CONTEXT_API_TOKEN"]

human_message_prompt = HumanMessagePromptTemplate(
prompt=PromptTemplate(
template="What is a good name for a company that makes {product}?",
input_variables=["product"],
)
)
chat_prompt_template = ChatPromptTemplate.from_messages([human_message_prompt])
callback = ContextCallbackHandler(token)
chat = ChatOpenAI(temperature=0.9, callbacks=[callback])
chain = LLMChain(llm=chat, prompt=chat_prompt_template, callbacks=[callback])
print(chain.run("colorful socks"))


This example shows how one can track the following while calling OpenAI models via LangChain and Infino:
# Set your key here.
# os.environ["OPENAI_API_KEY"] = "YOUR_API_KEY"

# Create callback handler. This logs latency, errors, token usage, prompts as well as prompt responses to Infino.
handler = InfinoCallbackHandler(
model_id="test_openai", model_version="0.1", verbose=False
)

# Create LLM.
llm = OpenAI(temperature=0.1)

# Number of questions to ask the OpenAI model. We limit to a short number here to save $$ while running this demo.
num_questions = 10

questions = questions[0:num_questions]
for question in questions:
print(question)


# We send the question to OpenAI API, with Infino callback.
llm_result = llm.generate([question], callbacks=[handler])
print(llm_result)
In what country is Normandy located?
generations=[[Generation(text='\n\nNormandy is located in France.', generation_info={'finish_reason': 'stop', 'logprobs': None})]] llm_output={'token_usage': {'total_tokens': 16, 'completion_tokens': 9, 'prompt_tokens': 7}, 'model_name': 'text-davinci-003'} run=RunInfo(run_id=UUID('8de21639-acec-4bd1-a12d-8124de1e20da'))
When were the Normans in Normandy?
generations=[[Generation(text='\n\nThe Normans first settled in Normandy in the late 9th century.', generation_info={'finish_reason': 'stop', 'logprobs': None})]] llm_output={'token_usage': {'total_tokens': 24, 'completion_tokens': 16, 'prompt_tokens': 8}, 'model_name': 'text-davinci-003'} run=RunInfo(run_id=UUID('cf81fc86-250b-4e6e-9d92-2df3bebb019a'))
From which countries did the Norse originate?
generations=[[Generation(text='\n\nThe Norse originated from Scandinavia, which includes modern-day Norway, Sweden, and Denmark.', generation_info={'finish_reason': 'stop', 'logprobs': None})]] llm_output={'token_usage': {'total_tokens': 29, 'completion_tokens': 21, 'prompt_tokens': 8}, 'model_name': 'text-davinci-003'} run=RunInfo(run_id=UUID('50f42f5e-b4a4-411a-a049-f92cb573a74f'))
Who was the Norse leader?


Who was the Norse leader?
generations=[[Generation(text='\n\nThe most famous Norse leader was the legendary Viking king Ragnar Lodbrok. He is believed to have lived in the 9th century and is renowned for his exploits in England and France.', generation_info={'finish_reason': 'stop', 'logprobs': None})]] llm_output={'token_usage': {'total_tokens': 45, 'completion_tokens': 39, 'prompt_tokens': 6}, 'model_name': 'text-davinci-003'} run=RunInfo(run_id=UUID('e32f31cb-ddc9-4863-8e6e-cb7a281a0ada'))
What century did the Normans first gain their separate identity?
generations=[[Generation(text='\n\nThe Normans first gained their separate identity in the 11th century.', generation_info={'finish_reason': 'stop', 'logprobs': None})]] llm_output={'token_usage': {'total_tokens': 28, 'completion_tokens': 16, 'prompt_tokens': 12}, 'model_name': 'text-davinci-003'} run=RunInfo(run_id=UUID('da9d8f73-b3b3-4bc5-8495-da8b11462a51'))
Who gave their name to Normandy in the 1000's and 1100's
generations=[[Generation(text='\n\nThe Normans, a people from northern France, gave their name to Normandy in the 1000s and 1100s. The Normans were descended from Viking settlers who had come to the region in the late 800s.', generation_info={'finish_reason': 'stop', 'logprobs': None})]] llm_output={'token_usage': {'total_tokens': 58, 'completion_tokens': 45, 'prompt_tokens': 13}, 'model_name': 'text-davinci-003'} run=RunInfo(run_id=UUID('bb5829bf-b6a6-4429-adfa-414ac5be46e5'))
What is France a region of?


What is France a region of?
generations=[[Generation(text='\n\nFrance is a region of Europe.', generation_info={'finish_reason': 'stop', 'logprobs': None})]] llm_output={'token_usage': {'total_tokens': 16, 'completion_tokens': 9, 'prompt_tokens': 7}, 'model_name': 'text-davinci-003'} run=RunInfo(run_id=UUID('6943880b-b4e4-4c74-9ca1-8c03c10f7e9c'))
Who did King Charles III swear fealty to?
generations=[[Generation(text='\n\nKing Charles III swore fealty to Pope Innocent III.', generation_info={'finish_reason': 'stop', 'logprobs': None})]] llm_output={'token_usage': {'total_tokens': 23, 'completion_tokens': 13, 'prompt_tokens': 10}, 'model_name': 'text-davinci-003'} run=RunInfo(run_id=UUID('c91fd663-09e6-4d00-b746-4c7fd96f9ceb'))
When did the Frankish identity emerge?
generations=[[Generation(text='\n\nThe Frankish identity began to emerge in the late 5th century, when the Franks began to expand their power and influence in the region. The Franks were a Germanic tribe that had migrated to the area from the east and had established a kingdom in what is now modern-day France. The Franks were eventually able to establish a powerful kingdom that lasted until the 10th century.', generation_info={'finish_reason': 'stop', 'logprobs': None})]] llm_output={'token_usage': {'total_tokens': 86, 'completion_tokens': 78, 'prompt_tokens': 8}, 'model_name': 'text-davinci-003'} run=RunInfo(run_id=UUID('23f86775-e592-4cb8-baa3-46ebe74305b2'))
Who was the duke in the battle of Hastings?


Who was the duke in the battle of Hastings?
generations=[[Generation(text='\n\nThe Duke of Normandy, William the Conqueror, was the leader of the Norman forces at the Battle of Hastings in 1066.', generation_info={'finish_reason': 'stop', 'logprobs': None})]] llm_output={'token_usage': {'total_tokens': 39, 'completion_tokens': 28, 'prompt_tokens': 11}, 'model_name': 'text-davinci-003'} run=RunInfo(run_id=UUID('ad5b7984-8758-4d95-a5eb-ee56e0218f6b'))
We now use matplotlib to create graphs of latency, errors and tokens consumed.
# Helper function to create a graph using matplotlib.
def plot(data, title):
data = json.loads(data)


# Extract x and y values from the data
timestamps = [item["time"] for item in data]
dates = [dt.datetime.fromtimestamp(ts) for ts in timestamps]
y = [item["value"] for item in data]

plt.rcParams["figure.figsize"] = [6, 4]
plt.subplots_adjust(bottom=0.2)
plt.xticks(rotation=25)
ax = plt.gca()
xfmt = md.DateFormatter("%Y-%m-%d %H:%M:%S")
ax.xaxis.set_major_formatter(xfmt)

# Create the plot
plt.plot(dates, y)

# Set labels and title
plt.xlabel("Time")
plt.ylabel("Value")
plt.title(title)

plt.show()

response = client.search_ts("__name__", "latency", 0, int(time.time()))
plot(response.text, "Latency")

response = client.search_ts("__name__", "error", 0, int(time.time()))
plot(response.text, "Errors")

response = client.search_ts("__name__", "prompt_tokens", 0, int(time.time()))
plot(response.text, "Prompt Tokens")

response = client.search_ts("__name__", "completion_tokens", 0, int(time.time()))
plot(response.text, "Completion Tokens")

response = client.search_ts("__name__", "total_tokens", 0, int(time.time()))
plot(response.text, "Total Tokens")
# Search for a particular prompt text.
query = "normandy"
response = client.search_log(query, 0, int(time.time()))
print("Results for", query, ":", response.text)

print("===")

query = "king charles III"
response = client.search_log("king charles III", 0, int(time.time()))
print("Results for", query, ":", response.text)


Label Studio
Label Studio is an open-source data labeling platform that provides LangChain with flexibility when it comes to labeling data for fine-tuning large language models (LLMs). It also enables the preparation of custom training data and the collection and evaluation of responses through human feedback.
In this guide, you will learn how to connect a LangChain pipeline to Label Studio to:
Aggregate all input prompts, conversations, and responses in a single LabelStudio project. This consolidates all the data in one place for easier labeling and analysis.
Refine prompts and responses to create a dataset for supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF) scenarios. The labeled data can be used to further train the LLM to improve its performance.
Evaluate model responses through human feedback. LabelStudio provides an interface for humans to review and provide feedback on model responses, allowing evaluation and iteration.
Installation and setup
First install latest versions of Label Studio and Label Studio API client:
pip install -U label-studio label-studio-sdk openai
Next, run label-studio on the command line to start the local LabelStudio instance at http://localhost:8080. See the Label Studio installation guide for more options.
You'll need a token to make API calls.
Open your LabelStudio instance in your browser, go to Account & Settings > Access Token and copy the key.
Set environment variables with your LabelStudio URL, API key and OpenAI API key:
import os


os.environ['LABEL_STUDIO_URL'] = '<YOUR-LABEL-STUDIO-URL>' # e.g. http://localhost:8080
os.environ['LABEL_STUDIO_API_KEY'] = '<YOUR-LABEL-STUDIO-API-KEY>'
os.environ['OPENAI_API_KEY'] = '<YOUR-OPENAI-API-KEY>'
Collecting LLMs prompts and responses
The data used for labeling is stored in projects within Label Studio. Every project is identified by an XML configuration that details the specifications for input and output data.
Create a project that takes human input in text format and outputs an editable LLM response in a text area:
<View>
<Style>
.prompt-box {
background-color: white;
border-radius: 10px;
box-shadow: 0px 4px 6px rgba(0, 0, 0, 0.1);
padding: 20px;
}
</Style>
<View className="root">
<View className="prompt-box">
<Text name="prompt" value="$prompt"/>
</View>
<TextArea name="response" toName="prompt"
maxSubmissions="1" editable="true"
required="true"/>
</View>
<Header value="Rate the response:"/>
<Rating name="rating" toName="prompt"/>
</View>
To create a project in Label Studio, click on the "Create" button.
Enter a name for your project in the "Project Name" field, such as My Project.
Navigate to Labeling Setup > Custom Template and paste the XML configuration provided above.
You can collect input LLM prompts and output responses in a LabelStudio project, connecting it via LabelStudioCallbackHandler:
from langchain.llms import OpenAI
from langchain.callbacks import LabelStudioCallbackHandler


llm = OpenAI(
temperature=0,
callbacks=[
LabelStudioCallbackHandler(
project_name="My Project"
)]
)
print(llm("Tell me a joke"))
In the Label Studio, open My Project. You will see the prompts, responses, and metadata like the model name.
Collecting Chat model Dialogues
You can also track and display full chat dialogues in LabelStudio, with the ability to rate and modify the last response:
Open Label Studio and click on the "Create" button.
Enter a name for your project in the "Project Name" field, such as New Project with Chat.
Navigate to Labeling Setup > Custom Template and paste the following XML configuration:
<View>
<View className="root">
<Paragraphs name="dialogue"
value="$prompt"
layout="dialogue"
textKey="content"
nameKey="role"
granularity="sentence"/>
<Header value="Final response:"/>
<TextArea name="response" toName="dialogue"
maxSubmissions="1" editable="true"
required="true"/>
</View>
<Header value="Rate the response:"/>
<Rating name="rating" toName="dialogue"/>
</View>
from langchain.chat_models import ChatOpenAI
from langchain.schema import HumanMessage, SystemMessage
from langchain.callbacks import LabelStudioCallbackHandler


chat_llm = ChatOpenAI(callbacks=[
LabelStudioCallbackHandler(
mode="chat",
project_name="New Project with Chat",
)
])
llm_results = chat_llm([
SystemMessage(content="Always use a lot of emojis"),
HumanMessage(content="Tell me a joke")
])
In Label Studio, open "New Project with Chat". Click on a created task to view dialog history and edit/annotate responses.
Custom Labeling Configuration
You can modify the default labeling configuration in LabelStudio to add more target labels like response sentiment, relevance, and many other types annotator's feedback.
New labeling configuration can be added from UI: go to Settings > Labeling Interface and set up a custom configuration with additional tags like Choices for sentiment or Rating for relevance. Keep in mind that TextArea tag should be presented in any configuration to display the LLM responses.
Alternatively, you can specify the labeling configuration on the initial call before project creation:
ls = LabelStudioCallbackHandler(project_config='''
<View>
<Text name="prompt" value="$prompt"/>
<TextArea name="response" toName="prompt"/>
<TextArea name="user_feedback" toName="prompt"/>
<Rating name="rating" toName="prompt"/>
<Choices name="sentiment" toName="prompt">
<Choice value="Positive"/>
<Choice value="Negative"/>
</Choices>
</View>
''')
Note that if the project doesn't exist, it will be created with the specified labeling configuration.
Other parameters
The LabelStudioCallbackHandler accepts several optional parameters:
api_key - Label Studio API key. Overrides environmental variable LABEL_STUDIO_API_KEY.
url - Label Studio URL. Overrides LABEL_STUDIO_URL, default http://localhost:8080.
project_id - Existing Label Studio project ID. Overrides LABEL_STUDIO_PROJECT_ID. Stores data in this project.
project_name - Project name if project ID not specified. Creates a new project. Default is "LangChain-%Y-%m-%d" formatted with the current date.
project_config - custom labeling configuration
mode: use this shortcut to create target configuration from scratch:
"prompt" - Single prompt, single response. Default.
"chat" - Multi-turn chat mode.


LLMonitor is an open-source observability platform that provides cost tracking, user tracking and powerful agent tracing.
Create an account on llmonitor.com, create an App, and then copy the associated tracking id. Once you have it, set it as an environment variable by running:
If you'd prefer not to set an environment variable, you can pass the key directly when initializing the callback handler:
from langchain.llms import OpenAI
from langchain.chat_models import ChatOpenAI
from langchain.callbacks import LLMonitorCallbackHandler

handler = LLMonitorCallbackHandler(app_id="...")

llm = OpenAI(
callbacks=[handler],
)

chat = ChatOpenAI(
callbacks=[handler],
metadata={"userId": "123"}, # you can assign user ids to models in the metadata
)
from langchain.agents import load_tools, initialize_agent, AgentType
from langchain.llms import OpenAI
from langchain.callbacks import LLMonitorCallbackHandler

handler = LLMonitorCallbackHandler(app_id="...")

llm = OpenAI(temperature=0)
tools = load_tools(["serpapi", "llm-math"], llm=llm)
agent = initialize_agent(tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION)
agent.run(
"Who is Leo DiCaprio's girlfriend? What is her current age raised to the 0.43 power?",
callbacks=[handler],
metadata={
"agentName": "Leo DiCaprio's girlfriend", # you can assign a custom agent in the metadata
},
)
For any question or issue with integration you can reach out to the LLMonitor team on Discord or via email.


PromptLayer
PromptLayer is a an LLM observability platform that lets you visualize requests, version prompts, and track usage. In this guide we will go over how to setup the PromptLayerCallbackHandler.
While PromptLayer does have LLMs that integrate directly with LangChain (e.g. PromptLayerOpenAI), this callback is the recommended way to integrate PromptLayer with LangChain.
See our docs for more information.
Installation and Setup
pip install promptlayer --upgrade
Getting API Credentials
If you do not have a PromptLayer account, create one on promptlayer.com. Then get an API key by clicking on the settings cog in the navbar and set it as an environment variabled called PROMPTLAYER_API_KEY
Usage
Getting started with PromptLayerCallbackHandler is fairly simple, it takes two optional arguments:
pl_tags - an optional list of strings that will be tracked as tags on PromptLayer.
pl_id_callback - an optional function that will take promptlayer_request_id as an argument. This ID can be used with all of PromptLayer's tracking features to track, metadata, scores, and prompt usage.
Simple OpenAI Example
In this simple example we use PromptLayerCallbackHandler with ChatOpenAI. We add a PromptLayer tag named chatopenai
import promptlayer # Don't forget this 🍰
from langchain.callbacks import PromptLayerCallbackHandler
from langchain.chat_models import ChatOpenAI
from langchain.schema import (
HumanMessage,
)

chat_llm = ChatOpenAI(
temperature=0,
callbacks=[PromptLayerCallbackHandler(pl_tags=["chatopenai"])],
)
llm_results = chat_llm(
[
HumanMessage(content="What comes after 1,2,3 ?"),
HumanMessage(content="Tell me another joke?"),
]
)
print(llm_results)
GPT4All Example
import promptlayer # Don't forget this 🍰
from langchain.callbacks import PromptLayerCallbackHandler

from langchain.llms import GPT4All

model = GPT4All(model="./models/gpt4all-model.bin", n_ctx=512, n_threads=8)


response = model(
"Once upon a time, ",
callbacks=[PromptLayerCallbackHandler(pl_tags=["langchain", "gpt4all"])],
)
Full Featured Example
In this example we unlock more of the power of PromptLayer.
PromptLayer allows you to visually create, version, and track prompt templates. Using the Prompt Registry, we can programatically fetch the prompt template called example.
We also define a pl_id_callback function which takes in the promptlayer_request_id and logs a score, metadata and links the prompt template used. Read more about tracking on our docs.
import promptlayer # Don't forget this 🍰
from langchain.callbacks import PromptLayerCallbackHandler
from langchain.llms import OpenAI

def pl_id_callback(promptlayer_request_id):
print("prompt layer id ", promptlayer_request_id)
promptlayer.track.score(
request_id=promptlayer_request_id, score=100
) # score is an integer 0-100
promptlayer.track.metadata(
request_id=promptlayer_request_id, metadata={"foo": "bar"}
) # metadata is a dictionary of key value pairs that is tracked on PromptLayer
promptlayer.track.prompt(
request_id=promptlayer_request_id,
prompt_name="example",
prompt_input_variables={"product": "toasters"},
version=1,
) # link the request to a prompt template

openai_llm = OpenAI(
model_name="text-davinci-002",
callbacks=[PromptLayerCallbackHandler(pl_id_callback=pl_id_callback)],
)

example_prompt = promptlayer.prompts.get("example", version=1, langchain=True)
openai_llm(example_prompt.format(product="toasters"))
That is all it takes! After setup all your requests will show up on the PromptLayer dashboard. This callback also works with any LLM implemented on LangChain.


Discord
This notebook shows how to create your own chat loader that works on copy-pasted messages (from dms) to a list of LangChain messages.
The process has four steps:
Create the chat .txt file by copying chats from the Discord app and pasting them in a file on your local computer
Copy the chat loader definition from below to a local file.
Initialize the DiscordChatLoader with the file path pointed to the text file.
Call loader.load() (or loader.lazy_load()) to perform the conversion.
1. Creat message dump
Currently (2023/08/23) this loader only supports .txt files in the format generated by copying messages in the app to your clipboard and pasting in a file. Below is an example.
discord_chats.txt
talkingtower — 08/15/2023 11:10 AM
Love music! Do you like jazz?
reporterbob — 08/15/2023 9:27 PM
Yes! Jazz is fantastic. Ever heard this one?
Website
Listen to classic jazz track...

talkingtower — Yesterday at 5:03 AM
Indeed! Great choice. 🎷
reporterbob — Yesterday at 5:23 AM
Thanks! How about some virtual sightseeing?
Website
Virtual tour of famous landmarks...

talkingtower — Today at 2:38 PM
Sounds fun! Let's explore.
reporterbob — Today at 2:56 PM
Enjoy the tour! See you around.
talkingtower — Today at 3:00 PM
Thank you! Goodbye! 👋
reporterbob — Today at 3:02 PM
Farewell! Happy exploring.
2. Define chat loader
LangChain currently does not support
import logging
import re
from typing import Iterator, List

from langchain import schema
from langchain.chat_loaders import base as chat_loaders

logger = logging.getLogger()

class DiscordChatLoader(chat_loaders.BaseChatLoader):

def __init__(self, path: str):
"""
Initialize the Discord chat loader.


def __init__(self, path: str):
"""
Initialize the Discord chat loader.

Args:
path: Path to the exported Discord chat text file.
"""
self.path = path
self._message_line_regex = re.compile(
r"(.+?) — (\w{3,9} \d{1,2}(?:st|nd|rd|th)?(?:, \d{4})? \d{1,2}:\d{2} (?:AM|PM)|Today at \d{1,2}:\d{2} (?:AM|PM)|Yesterday at \d{1,2}:\d{2} (?:AM|PM))", # noqa
flags=re.DOTALL,
)

def _load_single_chat_session_from_txt(
self, file_path: str
) -> chat_loaders.ChatSession:
"""
Load a single chat session from a text file.

Args:
file_path: Path to the text file containing the chat messages.

Returns:
A `ChatSession` object containing the loaded chat messages.
"""
with open(file_path, "r", encoding="utf-8") as file:
lines = file.readlines()


results: List[schema.BaseMessage] = []
current_sender = None
current_timestamp = None
current_content = []
for line in lines:
if re.match(
r".+? — (\d{2}/\d{2}/\d{4} \d{1,2}:\d{2} (?:AM|PM)|Today at \d{1,2}:\d{2} (?:AM|PM)|Yesterday at \d{1,2}:\d{2} (?:AM|PM))", # noqa
line,
):
if current_sender and current_content:
results.append(
schema.HumanMessage(
content="".join(current_content).strip(),
additional_kwargs={
"sender": current_sender,
"events": [{"message_time": current_timestamp}],
},
)
)
current_sender, current_timestamp = line.split(" — ")[:2]
current_content = [
line[len(current_sender) + len(current_timestamp) + 4 :].strip()
]
elif re.match(r"\[\d{1,2}:\d{2} (?:AM|PM)\]", line.strip()):
results.append(
schema.HumanMessage(
content="".join(current_content).strip(),
additional_kwargs={
"sender": current_sender,
"events": [{"message_time": current_timestamp}],
},
)
)
current_timestamp = line.strip()[1:-1]
current_content = []
else:
current_content.append("\n" + line.strip())

if current_sender and current_content:
results.append(
schema.HumanMessage(
content="".join(current_content).strip(),
additional_kwargs={
"sender": current_sender,
"events": [{"message_time": current_timestamp}],
},
)
)

return chat_loaders.ChatSession(messages=results)

def lazy_load(self) -> Iterator[chat_loaders.ChatSession]:
"""
Lazy load the messages from the chat file and yield them in the required format.


Yields:
A `ChatSession` object containing the loaded chat messages.
"""
yield self._load_single_chat_session_from_txt(self.path)
2. Create loader
We will point to the file we just wrote to disk.
loader = DiscordChatLoader(
path="./discord_chats.txt",
)
3. Load Messages
Assuming the format is correct, the loader will convert the chats to langchain messages.
from typing import List
from langchain.chat_loaders.base import ChatSession
from langchain.chat_loaders.utils import (
map_ai_messages,
merge_chat_runs,
)


raw_messages = loader.lazy_load()
# Merge consecutive messages from the same sender into a single message
merged_messages = merge_chat_runs(raw_messages)
# Convert messages from "talkingtower" to AI messages
messages: List[ChatSession] = list(map_ai_messages(merged_messages, sender="talkingtower"))
[{'messages': [AIMessage(content='Love music! Do you like jazz?', additional_kwargs={'sender': 'talkingtower', 'events': [{'message_time': '08/15/2023 11:10 AM\n'}]}, example=False),
HumanMessage(content='Yes! Jazz is fantastic. Ever heard this one?\nWebsite\nListen to classic jazz track...', additional_kwargs={'sender': 'reporterbob', 'events': [{'message_time': '08/15/2023 9:27 PM\n'}]}, example=False),
AIMessage(content='Indeed! Great choice. 🎷', additional_kwargs={'sender': 'talkingtower', 'events': [{'message_time': 'Yesterday at 5:03 AM\n'}]}, example=False),
HumanMessage(content='Thanks! How about some virtual sightseeing?\nWebsite\nVirtual tour of famous landmarks...', additional_kwargs={'sender': 'reporterbob', 'events': [{'message_time': 'Yesterday at 5:23 AM\n'}]}, example=False),
AIMessage(content="Sounds fun! Let's explore.", additional_kwargs={'sender': 'talkingtower', 'events': [{'message_time': 'Today at 2:38 PM\n'}]}, example=False),
HumanMessage(content='Enjoy the tour! See you around.', additional_kwargs={'sender': 'reporterbob', 'events': [{'message_time': 'Today at 2:56 PM\n'}]}, example=False),
AIMessage(content='Thank you! Goodbye! 👋', additional_kwargs={'sender': 'talkingtower', 'events': [{'message_time': 'Today at 3:00 PM\n'}]}, example=False),
HumanMessage(content='Farewell! Happy exploring.', additional_kwargs={'sender': 'reporterbob', 'events': [{'message_time': 'Today at 3:02 PM\n'}]}, example=False)]}]
Next Steps
You can then use these messages how you see fit, such as finetuning a model, few-shot example selection, or directly make predictions for the next message
from langchain.chat_models import ChatOpenAI

llm = ChatOpenAI()


llm = ChatOpenAI()

for chunk in llm.stream(messages[0]['messages']):
print(chunk.content, end="", flush=True)
Thank you! Have a wonderful day! 🌟


Facebook Messenger
This notebook shows how to load data from Facebook in a format you can finetune on. The overall steps are:
Download your messenger data to disk.
Create the Chat Loader and call loader.load() (or loader.lazy_load()) to perform the conversion.
Optionally use merge_chat_runs to combine message from the same sender in sequence, and/or map_ai_messages to convert messages from the specified sender to the "AIMessage" class. Once you've done this, call convert_messages_for_finetuning to prepare your data for fine-tuning.
Once this has been done, you can fine-tune your model. To do so you would complete the following steps:
Upload your messages to OpenAI and run a fine-tuning job.
Use the resulting model in your LangChain app!
Let's begin.
1. Download Data
To download your own messenger data, following instructions here. IMPORTANT - make sure to download them in JSON format (not HTML).
We are hosting an example dump at this google drive link that we will use in this walkthrough.
# This uses some example data
import requests
import zipfile

def download_and_unzip(url: str, output_path: str = 'file.zip') -> None:
file_id = url.split('/')[-2]
download_url = f'https://drive.google.com/uc?export=download&id={file_id}'

response = requests.get(download_url)
if response.status_code != 200:
print('Failed to download the file.')
return

with open(output_path, 'wb') as file:
file.write(response.content)
print(f'File {output_path} downloaded.')

with zipfile.ZipFile(output_path, 'r') as zip_ref:
zip_ref.extractall()
print(f'File {output_path} has been unzipped.')

# URL of the file to download
url = 'https://drive.google.com/file/d/1rh1s1o2i7B-Sk1v9o8KNgivLVGwJ-osV/view?usp=sharing'


# Download and unzip
download_and_unzip(url)
File file.zip downloaded.
File file.zip has been unzipped.
2. Create Chat Loader
We have 2 different FacebookMessengerChatLoader classes, one for an entire directory of chats, and one to load individual files. We
directory_path = "./hogwarts"
from langchain.chat_loaders.facebook_messenger import (
SingleFileFacebookMessengerChatLoader,
FolderFacebookMessengerChatLoader,
)
loader = SingleFileFacebookMessengerChatLoader(
path="./hogwarts/inbox/HermioneGranger/messages_Hermione_Granger.json",
)
chat_session = loader.load()[0]
chat_session["messages"][:3]
[HumanMessage(content="Hi Hermione! How's your summer going so far?", additional_kwargs={'sender': 'Harry Potter'}, example=False),
HumanMessage(content="Harry! Lovely to hear from you. My summer is going well, though I do miss everyone. I'm spending most of my time going through my books and researching fascinating new topics. How about you?", additional_kwargs={'sender': 'Hermione Granger'}, example=False),
HumanMessage(content="I miss you all too. The Dursleys are being their usual unpleasant selves but I'm getting by. At least I can practice some spells in my room without them knowing. Let me know if you find anything good in your researching!", additional_kwargs={'sender': 'Harry Potter'}, example=False)]
loader = FolderFacebookMessengerChatLoader(
path="./hogwarts",
)
chat_sessions = loader.load()
len(chat_sessions)
3. Prepare for fine-tuning
Calling load() returns all the chat messages we could extract as human messages. When conversing with chat bots, conversations typically follow a more strict alternating dialogue pattern relative to real conversations.
You can choose to merge message "runs" (consecutive messages from the same sender) and select a sender to represent the "AI". The fine-tuned LLM will learn to generate these AI messages.
from langchain.chat_loaders.utils import (
merge_chat_runs,
map_ai_messages,
)
merged_sessions = merge_chat_runs(chat_sessions)
alternating_sessions = list(map_ai_messages(merged_sessions, "Harry Potter"))
# Now all of Harry Potter's messages will take the AI message class


# Now all of Harry Potter's messages will take the AI message class
# which maps to the 'assistant' role in OpenAI's training format
alternating_sessions[0]['messages'][:3]
[AIMessage(content="Professor Snape, I was hoping I could speak with you for a moment about something that's been concerning me lately.", additional_kwargs={'sender': 'Harry Potter'}, example=False),
HumanMessage(content="What is it, Potter? I'm quite busy at the moment.", additional_kwargs={'sender': 'Severus Snape'}, example=False),
AIMessage(content="I apologize for the interruption, sir. I'll be brief. I've noticed some strange activity around the school grounds at night. I saw a cloaked figure lurking near the Forbidden Forest last night. I'm worried someone may be plotting something sinister.", additional_kwargs={'sender': 'Harry Potter'}, example=False)]
Now we can convert to OpenAI format dictionaries
from langchain.adapters.openai import convert_messages_for_finetuning
training_data = convert_messages_for_finetuning(alternating_sessions)
print(f"Prepared {len(training_data)} dialogues for training")
Prepared 9 dialogues for training
[{'role': 'assistant',
'content': "Professor Snape, I was hoping I could speak with you for a moment about something that's been concerning me lately."},
{'role': 'user',
'content': "What is it, Potter? I'm quite busy at the moment."},
{'role': 'assistant',
'content': "I apologize for the interruption, sir. I'll be brief. I've noticed some strange activity around the school grounds at night. I saw a cloaked figure lurking near the Forbidden Forest last night. I'm worried someone may be plotting something sinister."}]
OpenAI currently requires at least 10 training examples for a fine-tuning job, though they recommend between 50-100 for most tasks. Since we only have 9 chat sessions, we can subdivide them (optionally with some overlap) so that each training example is comprised of a portion of a whole conversation.
Facebook chat sessions (1 per person) often span multiple days and conversations, so the long-range dependencies may not be that important to model anyhow.
# Our chat is alternating, we will make each datapoint a group of 8 messages,
# with 2 messages overlapping


# with 2 messages overlapping
chunk_size = 8
overlap = 2


training_examples = [
conversation_messages[i: i + chunk_size]
for conversation_messages in training_data
for i in range(
0, len(conversation_messages) - chunk_size + 1,
chunk_size - overlap)
]

len(training_examples)
4. Fine-tune the model
It's time to fine-tune the model. Make sure you have openai installed and have set your OPENAI_API_KEY appropriately
# %pip install -U openai --quiet
import json
from io import BytesIO
import time

import openai

# We will write the jsonl file in memory
my_file = BytesIO()
for m in training_examples:
my_file.write((json.dumps({"messages": m}) + "\n").encode('utf-8'))

my_file.seek(0)
training_file = openai.File.create(
file=my_file,
purpose='fine-tune'
)


# OpenAI audits each training file for compliance reasons.
# This make take a few minutes
status = openai.File.retrieve(training_file.id).status
start_time = time.time()
while status != "processed":
print(f"Status=[{status}]... {time.time() - start_time:.2f}s", end="\r", flush=True)
time.sleep(5)
status = openai.File.retrieve(training_file.id).status
print(f"File {training_file.id} ready after {time.time() - start_time:.2f} seconds.")
File file-zCyNBeg4snpbBL7VkvsuhCz8 ready afer 30.55 seconds.
With the file ready, it's time to kick off a training job.
job = openai.FineTuningJob.create(
training_file=training_file.id,
model="gpt-3.5-turbo",
)
Grab a cup of tea while your model is being prepared. This may take some time!
status = openai.FineTuningJob.retrieve(job.id).status
start_time = time.time()
while status != "succeeded":
print(f"Status=[{status}]... {time.time() - start_time:.2f}s", end="\r", flush=True)
time.sleep(5)
job = openai.FineTuningJob.retrieve(job.id)
status = job.status
Status=[running]... 908.87s
print(job.fine_tuned_model)
ft:gpt-3.5-turbo-0613:personal::7rDwkaOq
5. Use in LangChain
You can use the resulting model ID directly the ChatOpenAI model class.
from langchain.chat_models import ChatOpenAI

model = ChatOpenAI(
model=job.fine_tuned_model,
temperature=1,
)
from langchain.prompts import ChatPromptTemplate
from langchain.schema.output_parser import StrOutputParser

prompt = ChatPromptTemplate.from_messages(
[
("human", "{input}"),
]
)

chain = prompt | model | StrOutputParser()
for tok in chain.stream({"input": "What classes are you taking?"}):
print(tok, end="", flush=True)
The usual - Potions, Transfiguration, Defense Against the Dark Arts. What about you?


GMail
This loader goes over how to load data from GMail. There are many ways you could want to load data from GMail. This loader is currently fairly opionated in how to do so. The way it does it is it first looks for all messages that you have sent. It then looks for messages where you are responding to a previous email. It then fetches that previous email, and creates a training example of that email, followed by your email.
Note that there are clear limitations here. For example, all examples created are only looking at the previous email for context.
To use:
Set up a Google Developer Account: Go to the Google Developer Console, create a project, and enable the Gmail API for that project. This will give you a credentials.json file that you'll need later.
Install the Google Client Library: Run the following command to install the Google Client Library:
pip install --upgrade google-auth google-auth-oauthlib google-auth-httplib2 google-api-python-client
import os.path
import base64
import json
import re
import time
from google.auth.transport.requests import Request
from google.oauth2.credentials import Credentials
from google_auth_oauthlib.flow import InstalledAppFlow
from googleapiclient.discovery import build
import logging
import requests

SCOPES = ['https://www.googleapis.com/auth/gmail.readonly']


SCOPES = ['https://www.googleapis.com/auth/gmail.readonly']

creds = None
# The file token.json stores the user's access and refresh tokens, and is
# created automatically when the authorization flow completes for the first
# time.
if os.path.exists('email_token.json'):
creds = Credentials.from_authorized_user_file('email_token.json', SCOPES)
# If there are no (valid) credentials available, let the user log in.
if not creds or not creds.valid:
if creds and creds.expired and creds.refresh_token:
creds.refresh(Request())
else:
flow = InstalledAppFlow.from_client_secrets_file(
# your creds file here. Please create json file as here https://cloud.google.com/docs/authentication/getting-started
'creds.json', SCOPES)
creds = flow.run_local_server(port=0)
# Save the credentials for the next run
with open('email_token.json', 'w') as token:
token.write(creds.to_json())
from langchain.chat_loaders.gmail import GMailLoader
loader = GMailLoader(creds=creds, n=3)
# Sometimes there can be errors which we silently ignore
len(data)
from langchain.chat_loaders.utils import (
map_ai_messages,
)
# This makes messages sent by hchase@langchain.com the AI Messages
# This means you will train an LLM to predict as if it's responding as hchase
training_data = list(map_ai_messages(data, sender="Harrison Chase <hchase@langchain.com>"))


Slack
This notebook shows how to use the Slack chat loader. This class helps map exported slack conversations to LangChain chat messages.
The process has three steps:
Export the desired conversation thread by following the instructions here.
Create the SlackChatLoader with the file path pointed to the json file or directory of JSON files
Call loader.load() (or loader.lazy_load()) to perform the conversion. Optionally use merge_chat_runs to combine message from the same sender in sequence, and/or map_ai_messages to convert messages from the specified sender to the "AIMessage" class.
1. Creat message dump
Currently (2023/08/23) this loader best supports a zip directory of files in the format generated by exporting your a direct message converstion from Slack. Follow up-to-date instructions from slack on how to do so.
We have an example in the LangChain repo.
import requests

permalink = "https://raw.githubusercontent.com/langchain-ai/langchain/342087bdfa3ac31d622385d0f2d09cf5e06c8db3/libs/langchain/tests/integration_tests/examples/slack_export.zip"
response = requests.get(permalink)
with open("slack_dump.zip", "wb") as f:
f.write(response.content)
2. Create the Chat Loader
Provide the loader with the file path to the zip directory. You can optionally specify the user id that maps to an ai message as well an configure whether to merge message runs.
from langchain.chat_loaders.slack import SlackChatLoader
loader = SlackChatLoader(
path="slack_dump.zip",
)
3. Load messages
The load() (or lazy_load) methods return a list of "ChatSessions" that currently just contain a list of messages per loaded conversation.
from typing import List
from langchain.chat_loaders.base import ChatSession
from langchain.chat_loaders.utils import (
map_ai_messages,
merge_chat_runs,
)


raw_messages = loader.lazy_load()
# Merge consecutive messages from the same sender into a single message
merged_messages = merge_chat_runs(raw_messages)
# Convert messages from "U0500003428" to AI messages
messages: List[ChatSession] = list(map_ai_messages(merged_messages, sender="U0500003428"))
Next Steps
You can then use these messages how you see fit, such as finetuning a model, few-shot example selection, or directly make predictions for the next message.
from langchain.chat_models import ChatOpenAI

llm = ChatOpenAI()

for chunk in llm.stream(messages[1]['messages']):
print(chunk.content, end="", flush=True)
Hi,

I hope you're doing well. I wanted to reach out and ask if you'd be available to meet up for coffee sometime next week. I'd love to catch up and hear about what's been going on in your life. Let me know if you're interested and we can find a time that works for both of us.

Looking forward to hearing from you!

Best, [Your Name]


iMessage
This notebook shows how to use the iMessage chat loader. This class helps convert iMessage conversations to LangChain chat messages.
On MacOS, iMessage stores conversations in a sqlite database at ~/Library/Messages/chat.db (at least for macOS Ventura 13.4). The IMessageChatLoader loads from this database file.
Create the IMessageChatLoader with the file path pointed to chat.db database you'd like to process.
Call loader.load() (or loader.lazy_load()) to perform the conversion. Optionally use merge_chat_runs to combine message from the same sender in sequence, and/or map_ai_messages to convert messages from the specified sender to the "AIMessage" class.
1. Access Chat DB
It's likely that your terminal is denied access to ~/Library/Messages. To use this class, you can copy the DB to an accessible directory (e.g., Documents) and load from there. Alternatively (and not recommended), you can grant full disk access for your terminal emulator in System Settings > Securityand Privacy > Full Disk Access.
We have created an example database you can use at this linked drive file.
# This uses some example data
import requests

def download_drive_file(url: str, output_path: str = 'chat.db') -> None:
file_id = url.split('/')[-2]
download_url = f'https://drive.google.com/uc?export=download&id={file_id}'

response = requests.get(download_url)
if response.status_code != 200:
print('Failed to download the file.')
return

with open(output_path, 'wb') as file:
file.write(response.content)
print(f'File {output_path} downloaded.')

url = 'https://drive.google.com/file/d/1NebNKqTA2NXApCmeH6mu0unJD2tANZzo/view?usp=sharing'


# Download file to chat.db
download_drive_file(url)
2. Create the Chat Loader
Provide the loader with the file path to the zip directory. You can optionally specify the user id that maps to an ai message as well an configure whether to merge message runs.
from langchain.chat_loaders.imessage import IMessageChatLoader
loader = IMessageChatLoader(
path="./chat.db",
)
3. Load messages
The load() (or lazy_load) methods return a list of "ChatSessions" that currently just contain a list of messages per loaded conversation. All messages are mapped to "HumanMessage" objects to start.
You can optionally choose to merge message "runs" (consecutive messages from the same sender) and select a sender to represent the "AI". The fine-tuned LLM will learn to generate these AI messages.
from typing import List
from langchain.chat_loaders.base import ChatSession
from langchain.chat_loaders.utils import (
map_ai_messages,
merge_chat_runs,
)


raw_messages = loader.lazy_load()
# Merge consecutive messages from the same sender into a single message
merged_messages = merge_chat_runs(raw_messages)
# Convert messages from "Tortoise" to AI messages. Do you have a guess who these conversations are between?
chat_sessions: List[ChatSession] = list(map_ai_messages(merged_messages, sender="Tortoise"))
# Now all of the Tortoise's messages will take the AI message class
# which maps to the 'assistant' role in OpenAI's training format
alternating_sessions[0]['messages'][:3]
[AIMessage(content="Slow and steady, that's my motto.", additional_kwargs={'message_time': 1693182723, 'sender': 'Tortoise'}, example=False),
HumanMessage(content='Speed is key!', additional_kwargs={'message_time': 1693182753, 'sender': 'Hare'}, example=False),
AIMessage(content='A balanced approach is more reliable.', additional_kwargs={'message_time': 1693182783, 'sender': 'Tortoise'}, example=False)]
3. Prepare for fine-tuning
Now it's time to convert our chat messages to OpenAI dictionaries. We can use the convert_messages_for_finetuning utility to do so.
from langchain.adapters.openai import convert_messages_for_finetuning
training_data = convert_messages_for_finetuning(alternating_sessions)
print(f"Prepared {len(training_data)} dialogues for training")
Prepared 10 dialogues for training
4. Fine-tune the model
It's time to fine-tune the model. Make sure you have openai installed and have set your OPENAI_API_KEY appropriately
# %pip install -U openai --quiet
import json
from io import BytesIO
import time

import openai

# We will write the jsonl file in memory
my_file = BytesIO()
for m in training_data:
my_file.write((json.dumps({"messages": m}) + "\n").encode('utf-8'))

my_file.seek(0)
training_file = openai.File.create(
file=my_file,
purpose='fine-tune'
)


# OpenAI audits each training file for compliance reasons.
# This make take a few minutes
status = openai.File.retrieve(training_file.id).status
start_time = time.time()
while status != "processed":
print(f"Status=[{status}]... {time.time() - start_time:.2f}s", end="\r", flush=True)
time.sleep(5)
status = openai.File.retrieve(training_file.id).status
print(f"File {training_file.id} ready after {time.time() - start_time:.2f} seconds.")
File file-zHIgf4r8LltZG3RFpkGd4Sjf ready after 10.19 seconds.
With the file ready, it's time to kick off a training job.
job = openai.FineTuningJob.create(
training_file=training_file.id,
model="gpt-3.5-turbo",
)
Grab a cup of tea while your model is being prepared. This may take some time!
status = openai.FineTuningJob.retrieve(job.id).status
start_time = time.time()
while status != "succeeded":
print(f"Status=[{status}]... {time.time() - start_time:.2f}s", end="\r", flush=True)
time.sleep(5)
job = openai.FineTuningJob.retrieve(job.id)
status = job.status
Status=[running]... 524.95s
print(job.fine_tuned_model)
ft:gpt-3.5-turbo-0613:personal::7sKoRdlz
5. Use in LangChain
You can use the resulting model ID directly the ChatOpenAI model class.
from langchain.chat_models import ChatOpenAI

model = ChatOpenAI(
model=job.fine_tuned_model,
temperature=1,
)
from langchain.prompts import ChatPromptTemplate
from langchain.schema.output_parser import StrOutputParser

prompt = ChatPromptTemplate.from_messages(
[
("system", "You are speaking to hare."),
("human", "{input}"),
]
)

chain = prompt | model | StrOutputParser()
for tok in chain.stream({"input": "What's the golden thread?"}):
print(tok, end="", flush=True)
A symbol of interconnectedness.


Telegram
This notebook shows how to use the Telegram chat loader. This class helps map exported Telegram conversations to LangChain chat messages.
The process has three steps:
Export the chat .txt file by copying chats from the Discord app and pasting them in a file on your local computer
Create the TelegramChatLoader with the file path pointed to the json file or directory of JSON files
Call loader.load() (or loader.lazy_load()) to perform the conversion. Optionally use merge_chat_runs to combine message from the same sender in sequence, and/or map_ai_messages to convert messages from the specified sender to the "AIMessage" class.
1. Creat message dump
Currently (2023/08/23) this loader best supports json files in the format generated by exporting your chat history from the Telegram Desktop App.
Important: There are 'lite' versions of telegram such as "Telegram for MacOS" that lack the export functionality. Please make sure you use the correct app to export the file.
To make the export:
Download and open telegram desktop
Select a conversation
Navigate to the conversation settings (currently the three dots in the top right corner)
Click "Export Chat History"
Unselect photos and other media. Select "Machine-readable JSON" format to export.
An example is below:
telegram_conversation.json
{
"name": "Jiminy",
"type": "personal_chat",
"id": 5965280513,
"messages": [
{
"id": 1,
"type": "message",
"date": "2023-08-23T13:11:23",
"date_unixtime": "1692821483",
"from": "Jiminy Cricket",
"from_id": "user123450513",
"text": "You better trust your conscience",
"text_entities": [
{
"type": "plain",
"text": "You better trust your conscience"
}
]
},
{
"id": 2,
"type": "message",
"date": "2023-08-23T13:13:20",
"date_unixtime": "1692821600",
"from": "Batman & Robin",
"from_id": "user6565661032",
"text": "What did you just say?",
"text_entities": [
{
"type": "plain",


"text_entities": [
{
"type": "plain",
"text": "What did you just say?"
}
]
}
]
}
2. Create the Chat Loader
All that's required is the file path. You can optionally specify the user name that maps to an ai message as well an configure whether to merge message runs.
from langchain.chat_loaders.telegram import TelegramChatLoader
loader = TelegramChatLoader(
path="./telegram_conversation.json",
)
3. Load messages
The load() (or lazy_load) methods return a list of "ChatSessions" that currently just contain a list of messages per loaded conversation.
from typing import List
from langchain.chat_loaders.base import ChatSession
from langchain.chat_loaders.utils import (
map_ai_messages,
merge_chat_runs,
)


raw_messages = loader.lazy_load()
# Merge consecutive messages from the same sender into a single message
merged_messages = merge_chat_runs(raw_messages)
# Convert messages from "Jiminy Cricket" to AI messages
messages: List[ChatSession] = list(map_ai_messages(merged_messages, sender="Jiminy Cricket"))
Next Steps
You can then use these messages how you see fit, such as finetuning a model, few-shot example selection, or directly make predictions for the next message
from langchain.chat_models import ChatOpenAI

llm = ChatOpenAI()

for chunk in llm.stream(messages[0]['messages']):
print(chunk.content, end="", flush=True)
I said, "You better trust your conscience."


This notebook shows how to use the WhatsApp chat loader. This class helps map exported Telegram conversations to LangChain chat messages.
To make the export of your WhatsApp conversation(s), complete the following steps:
whatsapp_chat.txt
[8/15/23, 9:12:33 AM] Dr. Feather: Messages and calls are end-to-end encrypted. No one outside of this chat, not even WhatsApp, can read or listen to them.
[8/15/23, 9:12:43 AM] Dr. Feather: I spotted a rare Hyacinth Macaw yesterday in the Amazon Rainforest. Such a magnificent creature!
[8/15/23, 9:12:48 AM] Dr. Feather: image omitted
[8/15/23, 9:13:15 AM] Jungle Jane: That's stunning! Were you able to observe its behavior?
[8/15/23, 9:13:23 AM] Dr. Feather: image omitted
[8/15/23, 9:14:02 AM] Dr. Feather: Yes, it seemed quite social with other macaws. They're known for their playful nature.
[8/15/23, 9:14:15 AM] Jungle Jane: How's the research going on parrot communication?
[8/15/23, 9:14:30 AM] Dr. Feather: image omitted
[8/15/23, 9:14:50 AM] Dr. Feather: It's progressing well. We're learning so much about how they use sound and color to communicate.
[8/15/23, 9:15:10 AM] Jungle Jane: That's fascinating! Can't wait to read your paper on it.
[8/15/23, 9:15:20 AM] Dr. Feather: Thank you! I'll send you a draft soon.
[8/15/23, 9:25:16 PM] Jungle Jane: Looking forward to it! Keep up the great work.
The WhatsAppChatLoader accepts the resulting zip file, unzipped directory, or the path to any of the chat .txt files therein.
Provide that as well as the user name you want to take on the role of "AI" when finetuning.


The load() (or lazy_load) methods return a list of "ChatSessions" that currently store the list of messages per loaded conversation.
[{'messages': [AIMessage(content='I spotted a rare Hyacinth Macaw yesterday in the Amazon Rainforest. Such a magnificent creature!', additional_kwargs={'sender': 'Dr. Feather', 'events': [{'message_time': '8/15/23, 9:12:43 AM'}]}, example=False),
HumanMessage(content="That's stunning! Were you able to observe its behavior?", additional_kwargs={'sender': 'Jungle Jane', 'events': [{'message_time': '8/15/23, 9:13:15 AM'}]}, example=False),
AIMessage(content="Yes, it seemed quite social with other macaws. They're known for their playful nature.", additional_kwargs={'sender': 'Dr. Feather', 'events': [{'message_time': '8/15/23, 9:14:02 AM'}]}, example=False),
HumanMessage(content="How's the research going on parrot communication?", additional_kwargs={'sender': 'Jungle Jane', 'events': [{'message_time': '8/15/23, 9:14:15 AM'}]}, example=False),
AIMessage(content="It's progressing well. We're learning so much about how they use sound and color to communicate.", additional_kwargs={'sender': 'Dr. Feather', 'events': [{'message_time': '8/15/23, 9:14:50 AM'}]}, example=False),
HumanMessage(content="That's fascinating! Can't wait to read your paper on it.", additional_kwargs={'sender': 'Jungle Jane', 'events': [{'message_time': '8/15/23, 9:15:10 AM'}]}, example=False),
AIMessage(content="Thank you! I'll send you a draft soon.", additional_kwargs={'sender': 'Dr. Feather', 'events': [{'message_time': '8/15/23, 9:15:20 AM'}]}, example=False),
HumanMessage(content='Looking forward to it! Keep up the great work.', additional_kwargs={'sender': 'Jungle Jane', 'events': [{'message_time': '8/15/23, 9:25:16 PM'}]}, example=False)]}]


You can then use these messages how you see fit, such as finetuning a model, few-shot example selection, or directly make predictions for the next message.


This notebook shows how to load chat messages from Twitter to finetune on. We do this by utilizing Apify.
First, use Apify to export tweets. An example
# Filter out tweets that reference other tweets, because it's a bit weird
tweets = [d["full_text"] for d in data if "t.co" not in d['full_text']]
# Create them as AI messages
messages = [AIMessage(content=t) for t in tweets]
# Add in a system message at the start
# TODO: we could try to extract the subject from the tweets, and put that in the system message.
system_message = {"role": "system", "content": "write a tweet"}
data = [[system_message, convert_message_to_dict(m)] for m in messages]


Beautiful Soup
Beautiful Soup offers fine-grained control over HTML content, enabling specific tag extraction, removal, and content cleaning.
It's suited for cases where you want to extract specific information and clean up the HTML content according to your needs.
For example, we can scrape text content within <p>, <li>, <div>, and <a> tags from the HTML content:
<p>: The paragraph tag. It defines a paragraph in HTML and is used to group together related sentences and/or phrases.
<li>: The list item tag. It is used within ordered (<ol>) and unordered (<ul>) lists to define individual items within the list.
<div>: The division tag. It is a block-level element used to group other inline or block-level elements.
<a>: The anchor tag. It is used to define hyperlinks.
from langchain.document_loaders import AsyncChromiumLoader
from langchain.document_transformers import BeautifulSoupTransformer

# Load HTML
loader = AsyncChromiumLoader(["https://www.wsj.com"])
html = loader.load()
# Transform
bs_transformer = BeautifulSoupTransformer()
docs_transformed = bs_transformer.transform_documents(html,tags_to_extract=["p", "li", "div", "a"])
docs_transformed[0].page_content[0:500]
'Conservative legal activists are challenging Amazon, Comcast and others using many of the same tools that helped kill affirmative-action programs in colleges.1,2099 min read U.S. stock indexes fell and government-bond prices climbed, after Moody’s lowered credit ratings for 10 smaller U.S. banks and said it was reviewing ratings for six larger ones. The Dow industrials dropped more than 150 points.3 min read Penn Entertainment’s Barstool Sportsbook app will be rebranded as ESPN Bet this fall as '


docai
from langchain.document_loaders.blob_loaders import Blob
from langchain.document_loaders.parsers import DocAIParser
DocAI is a Google Cloud platform to transform unstructured data from documents into structured data, making it easier to understand, analyze, and consume. You can read more about it: https://cloud.google.com/document-ai/docs/overview
First, you need to set up a GCS bucket and create your own OCR processor as described here: https://cloud.google.com/document-ai/docs/create-processor The GCS_OUTPUT_PATH should be a path to a folder on GCS (starting with gs://) and a processor name should look like projects/PROJECT_NUMBER/locations/LOCATION/processors/PROCESSOR_ID. You can get it either programmatically or copy from the Prediction endpoint section of the Processor details tab in the Google Cloud Console.
PROJECT = "PUT_SOMETHING_HERE"
GCS_OUTPUT_PATH = "PUT_SOMETHING_HERE"
PROCESSOR_NAME = "PUT_SOMETHING_HERE"
Now, let's create a parser:
parser = DocAIParser(location="us", processor_name=PROCESSOR_NAME, gcs_output_path=GCS_OUTPUT_PATH)
Let's go and parse an Alphabet's take from here: https://abc.xyz/assets/a7/5b/9e5ae0364b12b4c883f3cf748226/goog-exhibit-99-1-q1-2023-19.pdf. Copy it to your GCS bucket first, and adjust the path below.
blob = Blob(path="gs://vertex-pgt/examples/goog-exhibit-99-1-q1-2023-19.pdf")
docs = list(parser.lazy_parse(blob))
We'll get one document per page, 11 in total:
You can run end-to-end parsing of a blob one-by-one. If you have many documents, it might be a better approach to batch them together and maybe even detach parsing from handling the results of parsing.
operations = parser.docai_parse([blob])
print([op.operation.name for op in operations])
['projects/543079149601/locations/us/operations/16447136779727347991']
You can check whether operations are finished:
parser.is_running(operations)
And when they're finished, you can parse the results:
parser.is_running(operations)


And when they're finished, you can parse the results:
parser.is_running(operations)
results = parser.get_results(operations)
print(results[0])
DocAIParsingResults(source_path='gs://vertex-pgt/examples/goog-exhibit-99-1-q1-2023-19.pdf', parsed_path='gs://vertex-pgt/test/run1/16447136779727347991/0')
And now we can finally generate Documents from parsed results:
docs = list(parser.parse_from_results(results))


Doctran Extract Properties
We can extract useful features of documents using the Doctran library, which uses OpenAI's function calling feature to extract specific metadata.
Extracting metadata from documents is helpful for a variety of tasks, including:
Classification: classifying documents into different categories
Data mining: Extract structured data that can be used for data analysis
Style transfer: Change the way text is written to more closely match expected user input, improving vector search results
import json
from langchain.schema import Document
from langchain.document_transformers import DoctranPropertyExtractor
from dotenv import load_dotenv

load_dotenv()
Input
This is the document we'll extract properties from.
sample_text = """[Generated with ChatGPT]

Confidential Document - For Internal Use Only

Date: July 1, 2023

Subject: Updates and Discussions on Various Topics

Dear Team,

I hope this email finds you well. In this document, I would like to provide you with some important updates and discuss various topics that require our attention. Please treat the information contained herein as highly confidential.

Security and Privacy Measures
As part of our ongoing commitment to ensure the security and privacy of our customers' data, we have implemented robust measures across all our systems. We would like to commend John Doe (email: john.doe@example.com) from the IT department for his diligent work in enhancing our network security. Moving forward, we kindly remind everyone to strictly adhere to our data protection policies and guidelines. Additionally, if you come across any potential security risks or incidents, please report them immediately to our dedicated team at security@example.com.

HR Updates and Employee Benefits
Recently, we welcomed several new team members who have made significant contributions to their respective departments. I would like to recognize Jane Smith (SSN: 049-45-5928) for her outstanding performance in customer service. Jane has consistently received positive feedback from our clients. Furthermore, please remember that the open enrollment period for our employee benefits program is fast approaching. Should you have any questions or require assistance, please contact our HR representative, Michael Johnson (phone: 418-492-3850, email: michael.johnson@example.com).


Marketing Initiatives and Campaigns
Our marketing team has been actively working on developing new strategies to increase brand awareness and drive customer engagement. We would like to thank Sarah Thompson (phone: 415-555-1234) for her exceptional efforts in managing our social media platforms. Sarah has successfully increased our follower base by 20% in the past month alone. Moreover, please mark your calendars for the upcoming product launch event on July 15th. We encourage all team members to attend and support this exciting milestone for our company.

Research and Development Projects
In our pursuit of innovation, our research and development department has been working tirelessly on various projects. I would like to acknowledge the exceptional work of David Rodriguez (email: david.rodriguez@example.com) in his role as project lead. David's contributions to the development of our cutting-edge technology have been instrumental. Furthermore, we would like to remind everyone to share their ideas and suggestions for potential new projects during our monthly R&D brainstorming session, scheduled for July 10th.

Please treat the information in this document with utmost confidentiality and ensure that it is not shared with unauthorized individuals. If you have any questions or concerns regarding the topics discussed, please do not hesitate to reach out to me directly.

Thank you for your attention, and let's continue to work together to achieve our goals.

Best regards,

Jason Fan
Cofounder & CEO
Psychic
jason@psychic.dev
"""
print(sample_text)
[Generated with ChatGPT]

Confidential Document - For Internal Use Only

Date: July 1, 2023

Subject: Updates and Discussions on Various Topics

Dear Team,

I hope this email finds you well. In this document, I would like to provide you with some important updates and discuss various topics that require our attention. Please treat the information contained herein as highly confidential.

Security and Privacy Measures
As part of our ongoing commitment to ensure the security and privacy of our customers' data, we have implemented robust measures across all our systems. We would like to commend John Doe (email: john.doe@example.com) from the IT department for his diligent work in enhancing our network security. Moving forward, we kindly remind everyone to strictly adhere to our data protection policies and guidelines. Additionally, if you come across any potential security risks or incidents, please report them immediately to our dedicated team at security@example.com.


HR Updates and Employee Benefits
Recently, we welcomed several new team members who have made significant contributions to their respective departments. I would like to recognize Jane Smith (SSN: 049-45-5928) for her outstanding performance in customer service. Jane has consistently received positive feedback from our clients. Furthermore, please remember that the open enrollment period for our employee benefits program is fast approaching. Should you have any questions or require assistance, please contact our HR representative, Michael Johnson (phone: 418-492-3850, email: michael.johnson@example.com).

Marketing Initiatives and Campaigns
Our marketing team has been actively working on developing new strategies to increase brand awareness and drive customer engagement. We would like to thank Sarah Thompson (phone: 415-555-1234) for her exceptional efforts in managing our social media platforms. Sarah has successfully increased our follower base by 20% in the past month alone. Moreover, please mark your calendars for the upcoming product launch event on July 15th. We encourage all team members to attend and support this exciting milestone for our company.

Research and Development Projects
In our pursuit of innovation, our research and development department has been working tirelessly on various projects. I would like to acknowledge the exceptional work of David Rodriguez (email: david.rodriguez@example.com) in his role as project lead. David's contributions to the development of our cutting-edge technology have been instrumental. Furthermore, we would like to remind everyone to share their ideas and suggestions for potential new projects during our monthly R&D brainstorming session, scheduled for July 10th.

Please treat the information in this document with utmost confidentiality and ensure that it is not shared with unauthorized individuals. If you have any questions or concerns regarding the topics discussed, please do not hesitate to reach out to me directly.

Thank you for your attention, and let's continue to work together to achieve our goals.

Best regards,

Jason Fan
Cofounder & CEO
Psychic
jason@psychic.dev


Jason Fan
Cofounder & CEO
Psychic
jason@psychic.dev

documents = [Document(page_content=sample_text)]
properties = [
{
"name": "category",
"description": "What type of email this is.",
"type": "string",
"enum": ["update", "action_item", "customer_feedback", "announcement", "other"],
"required": True,
},
{
"name": "mentions",
"description": "A list of all people mentioned in this email.",
"type": "array",
"items": {
"name": "full_name",
"description": "The full name of the person mentioned.",
"type": "string",
},
"required": True,
},
{
"name": "eli5",
"description": "Explain this email to me like I'm 5 years old.",
"type": "string",
"required": True,
},
]
property_extractor = DoctranPropertyExtractor(properties=properties)
Output
After extracting properties from a document, the result will be returned as a new document with properties provided in the metadata
extracted_document = await property_extractor.atransform_documents(
documents, properties=properties
)
print(json.dumps(extracted_document[0].metadata, indent=2))
{
"extracted_properties": {
"category": "update",
"mentions": [
"John Doe",
"Jane Smith",
"Michael Johnson",
"Sarah Thompson",
"David Rodriguez",
"Jason Fan"
],
"eli5": "This is an email from the CEO, Jason Fan, giving updates about different areas in the company. He talks about new security measures and praises John Doe for his work. He also mentions new hires and praises Jane Smith for her work in customer service. The CEO reminds everyone about the upcoming benefits enrollment and says to contact Michael Johnson with any questions. He talks about the marketing team's work and praises Sarah Thompson for increasing their social media followers. There's also a product launch event on July 15th. Lastly, he talks about the research and development projects and praises David Rodriguez for his work. There's a brainstorming session on July 10th."
}
}


Documents used in a vector store knowledge base are typically stored in narrative or conversational format. However, most user queries are in question format. If we convert documents into Q&A format before vectorizing them, we can increase the liklihood of retrieving relevant documents, and decrease the liklihood of retrieving irrelevant documents.
We can accomplish this using the Doctran library, which uses OpenAI's function calling feature to "interrogate" documents.
See this notebook for benchmarks on vector similarity scores for various queries based on raw documents versus interrogated documents.
sample_text = """[Generated with ChatGPT]

Confidential Document - For Internal Use Only

Date: July 1, 2023

Subject: Updates and Discussions on Various Topics

Dear Team,

I hope this email finds you well. In this document, I would like to provide you with some important updates and discuss various topics that require our attention. Please treat the information contained herein as highly confidential.

Security and Privacy Measures
As part of our ongoing commitment to ensure the security and privacy of our customers' data, we have implemented robust measures across all our systems. We would like to commend John Doe (email: john.doe@example.com) from the IT department for his diligent work in enhancing our network security. Moving forward, we kindly remind everyone to strictly adhere to our data protection policies and guidelines. Additionally, if you come across any potential security risks or incidents, please report them immediately to our dedicated team at security@example.com.

HR Updates and Employee Benefits
Recently, we welcomed several new team members who have made significant contributions to their respective departments. I would like to recognize Jane Smith (SSN: 049-45-5928) for her outstanding performance in customer service. Jane has consistently received positive feedback from our clients. Furthermore, please remember that the open enrollment period for our employee benefits program is fast approaching. Should you have any questions or require assistance, please contact our HR representative, Michael Johnson (phone: 418-492-3850, email: michael.johnson@example.com).


Marketing Initiatives and Campaigns
Our marketing team has been actively working on developing new strategies to increase brand awareness and drive customer engagement. We would like to thank Sarah Thompson (phone: 415-555-1234) for her exceptional efforts in managing our social media platforms. Sarah has successfully increased our follower base by 20% in the past month alone. Moreover, please mark your calendars for the upcoming product launch event on July 15th. We encourage all team members to attend and support this exciting milestone for our company.

Research and Development Projects
In our pursuit of innovation, our research and development department has been working tirelessly on various projects. I would like to acknowledge the exceptional work of David Rodriguez (email: david.rodriguez@example.com) in his role as project lead. David's contributions to the development of our cutting-edge technology have been instrumental. Furthermore, we would like to remind everyone to share their ideas and suggestions for potential new projects during our monthly R&D brainstorming session, scheduled for July 10th.

Please treat the information in this document with utmost confidentiality and ensure that it is not shared with unauthorized individuals. If you have any questions or concerns regarding the topics discussed, please do not hesitate to reach out to me directly.

Thank you for your attention, and let's continue to work together to achieve our goals.

Best regards,


Best regards,

Jason Fan
Cofounder & CEO
Psychic
jason@psychic.dev
"""
print(sample_text)
After interrogating a document, the result will be returned as a new document with questions and answers provided in the metadata.
{
"questions_and_answers": [
{
"question": "What is the purpose of this document?",
"answer": "The purpose of this document is to provide important updates and discuss various topics that require the team's attention."
},
{
"question": "Who is responsible for enhancing the network security?",
"answer": "John Doe from the IT department is responsible for enhancing the network security."
},
{
"question": "Where should potential security risks or incidents be reported?",
"answer": "Potential security risks or incidents should be reported to the dedicated team at security@example.com."
},
{
"question": "Who has been recognized for outstanding performance in customer service?",
"answer": "Jane Smith has been recognized for her outstanding performance in customer service."
},
{
"question": "When is the open enrollment period for the employee benefits program?",
"answer": "The document does not specify the exact dates for the open enrollment period for the employee benefits program, but it mentions that it is fast approaching."
},
{
"question": "Who should be contacted for questions or assistance regarding the employee benefits program?",
"answer": "For questions or assistance regarding the employee benefits program, the HR representative, Michael Johnson, should be contacted."
},
{
"question": "Who has been acknowledged for managing the company's social media platforms?",
"answer": "Sarah Thompson has been acknowledged for managing the company's social media platforms."
},
{
"question": "When is the upcoming product launch event?",
"answer": "The upcoming product launch event is on July 15th."
},
{
"question": "Who has been recognized for their contributions to the development of the company's technology?",
"answer": "David Rodriguez has been recognized for his contributions to the development of the company's technology."
},
{
"question": "When is the monthly R&D brainstorming session?",
"answer": "The monthly R&D brainstorming session is scheduled for July 10th."
},
{
"question": "Who should be contacted for questions or concerns regarding the topics discussed in the document?",
"answer": "For questions or concerns regarding the topics discussed in the document, Jason Fan, the Cofounder & CEO, should be contacted."
}
]
}


Comparing documents through embeddings has the benefit of working across multiple languages. "Harrison says hello" and "Harrison dice hola" will occupy similar positions in the vector space because they have the same meaning semantically.
However, it can still be useful to use a LLM translate documents into other languages before vectorizing them. This is especially helpful when users are expected to query the knowledge base in different languages, or when state of the art embeddings models are not available for a given language.
We can accomplish this using the Doctran library, which uses OpenAI's function calling feature to translate documents between languages.
sample_text = """[Generated with ChatGPT]

Confidential Document - For Internal Use Only

Date: July 1, 2023

Subject: Updates and Discussions on Various Topics

Dear Team,

I hope this email finds you well. In this document, I would like to provide you with some important updates and discuss various topics that require our attention. Please treat the information contained herein as highly confidential.

Security and Privacy Measures
As part of our ongoing commitment to ensure the security and privacy of our customers' data, we have implemented robust measures across all our systems. We would like to commend John Doe (email: john.doe@example.com) from the IT department for his diligent work in enhancing our network security. Moving forward, we kindly remind everyone to strictly adhere to our data protection policies and guidelines. Additionally, if you come across any potential security risks or incidents, please report them immediately to our dedicated team at security@example.com.

HR Updates and Employee Benefits
Recently, we welcomed several new team members who have made significant contributions to their respective departments. I would like to recognize Jane Smith (SSN: 049-45-5928) for her outstanding performance in customer service. Jane has consistently received positive feedback from our clients. Furthermore, please remember that the open enrollment period for our employee benefits program is fast approaching. Should you have any questions or require assistance, please contact our HR representative, Michael Johnson (phone: 418-492-3850, email: michael.johnson@example.com).


Marketing Initiatives and Campaigns
Our marketing team has been actively working on developing new strategies to increase brand awareness and drive customer engagement. We would like to thank Sarah Thompson (phone: 415-555-1234) for her exceptional efforts in managing our social media platforms. Sarah has successfully increased our follower base by 20% in the past month alone. Moreover, please mark your calendars for the upcoming product launch event on July 15th. We encourage all team members to attend and support this exciting milestone for our company.

Research and Development Projects
In our pursuit of innovation, our research and development department has been working tirelessly on various projects. I would like to acknowledge the exceptional work of David Rodriguez (email: david.rodriguez@example.com) in his role as project lead. David's contributions to the development of our cutting-edge technology have been instrumental. Furthermore, we would like to remind everyone to share their ideas and suggestions for potential new projects during our monthly R&D brainstorming session, scheduled for July 10th.

Please treat the information in this document with utmost confidentiality and ensure that it is not shared with unauthorized individuals. If you have any questions or concerns regarding the topics discussed, please do not hesitate to reach out to me directly.

Thank you for your attention, and let's continue to work together to achieve our goals.

Best regards,

Jason Fan
Cofounder & CEO
Psychic
jason@psychic.dev
"""
After translating a document, the result will be returned as a new document with the page_content translated into the target language
[Generado con ChatGPT]

Documento confidencial - Solo para uso interno

Fecha: 1 de julio de 2023

Asunto: Actualizaciones y discusiones sobre varios temas

Estimado equipo,

Espero que este correo electrónico les encuentre bien. En este documento, me gustaría proporcionarles algunas actualizaciones importantes y discutir varios temas que requieren nuestra atención. Por favor, traten la información contenida aquí como altamente confidencial.


Medidas de seguridad y privacidad
Como parte de nuestro compromiso continuo para garantizar la seguridad y privacidad de los datos de nuestros clientes, hemos implementado medidas robustas en todos nuestros sistemas. Nos gustaría elogiar a John Doe (correo electrónico: john.doe@example.com) del departamento de TI por su diligente trabajo en mejorar nuestra seguridad de red. En adelante, recordamos amablemente a todos que se adhieran estrictamente a nuestras políticas y directrices de protección de datos. Además, si se encuentran con cualquier riesgo de seguridad o incidente potencial, por favor repórtelo inmediatamente a nuestro equipo dedicado en security@example.com.

Actualizaciones de RRHH y beneficios para empleados
Recientemente, dimos la bienvenida a varios nuevos miembros del equipo que han hecho contribuciones significativas a sus respectivos departamentos. Me gustaría reconocer a Jane Smith (SSN: 049-45-5928) por su sobresaliente rendimiento en el servicio al cliente. Jane ha recibido constantemente comentarios positivos de nuestros clientes. Además, recuerden que el período de inscripción abierta para nuestro programa de beneficios para empleados se acerca rápidamente. Si tienen alguna pregunta o necesitan asistencia, por favor contacten a nuestro representante de RRHH, Michael Johnson (teléfono: 418-492-3850, correo electrónico: michael.johnson@example.com).

Iniciativas y campañas de marketing
Nuestro equipo de marketing ha estado trabajando activamente en el desarrollo de nuevas estrategias para aumentar la conciencia de marca y fomentar la participación del cliente. Nos gustaría agradecer a Sarah Thompson (teléfono: 415-555-1234) por sus excepcionales esfuerzos en la gestión de nuestras plataformas de redes sociales. Sarah ha aumentado con éxito nuestra base de seguidores en un 20% solo en el último mes. Además, por favor marquen sus calendarios para el próximo evento de lanzamiento de producto el 15 de julio. Animamos a todos los miembros del equipo a asistir y apoyar este emocionante hito para nuestra empresa.


Proyectos de investigación y desarrollo
En nuestra búsqueda de la innovación, nuestro departamento de investigación y desarrollo ha estado trabajando incansablemente en varios proyectos. Me gustaría reconocer el excepcional trabajo de David Rodríguez (correo electrónico: david.rodriguez@example.com) en su papel de líder de proyecto. Las contribuciones de David al desarrollo de nuestra tecnología de vanguardia han sido fundamentales. Además, nos gustaría recordar a todos que compartan sus ideas y sugerencias para posibles nuevos proyectos durante nuestra sesión de lluvia de ideas de I+D mensual, programada para el 10 de julio.

Por favor, traten la información de este documento con la máxima confidencialidad y asegúrense de que no se comparte con personas no autorizadas. Si tienen alguna pregunta o inquietud sobre los temas discutidos, no duden en ponerse en contacto conmigo directamente.

Gracias por su atención, y sigamos trabajando juntos para alcanzar nuestros objetivos.

Saludos cordiales,

Jason Fan
Cofundador y CEO
Psychic
jason@psychic.dev


html2text
html2text is a Python script that converts a page of HTML into clean, easy-to-read plain ASCII text.
The ASCII also happens to be valid Markdown (a text-to-HTML format).
from langchain.document_loaders import AsyncHtmlLoader


urls = ["https://www.espn.com", "https://lilianweng.github.io/posts/2023-06-23-agent/"]
loader = AsyncHtmlLoader(urls)
docs = loader.load()
Fetching pages: 100%|############| 2/2 [00:00<00:00, 10.75it/s]
from langchain.document_transformers import Html2TextTransformer
urls = ["https://www.espn.com", "https://lilianweng.github.io/posts/2023-06-23-agent/"]
html2text = Html2TextTransformer()
docs_transformed = html2text.transform_documents(docs)
docs_transformed[0].page_content[1000:2000]
" * ESPNFC\n\n * X Games\n\n * SEC Network\n\n## ESPN Apps\n\n * ESPN\n\n * ESPN Fantasy\n\n## Follow ESPN\n\n * Facebook\n\n * Twitter\n\n * Instagram\n\n * Snapchat\n\n * YouTube\n\n * The ESPN Daily Podcast\n\n2023 FIFA Women's World Cup\n\n## Follow live: Canada takes on Nigeria in group stage of Women's World Cup\n\n2m\n\nEPA/Morgan Hancock\n\n## TOP HEADLINES\n\n * Snyder fined $60M over findings in investigation\n * NFL owners approve $6.05B sale of Commanders\n * Jags assistant comes out as gay in NFL milestone\n * O's alone atop East after topping slumping Rays\n * ACC's Phillips: Never condoned hazing at NU\n\n * Vikings WR Addison cited for driving 140 mph\n * 'Taking his time': Patient QB Rodgers wows Jets\n * Reyna got U.S. assurances after Berhalter rehire\n * NFL Future Power Rankings\n\n## USWNT AT THE WORLD CUP\n\n### USA VS. VIETNAM: 9 P.M. ET FRIDAY\n\n## How do you defend against Alex Morgan? Former opponents sound off\n\nThe U.S. forward is unstoppable at this level, scoring 121 goals and adding 49"
docs_transformed[1].page_content[1000:2000]


docs_transformed[1].page_content[1000:2000]
"t's brain,\ncomplemented by several key components:\n\n * **Planning**\n * Subgoal and decomposition: The agent breaks down large tasks into smaller, manageable subgoals, enabling efficient handling of complex tasks.\n * Reflection and refinement: The agent can do self-criticism and self-reflection over past actions, learn from mistakes and refine them for future steps, thereby improving the quality of final results.\n * **Memory**\n * Short-term memory: I would consider all the in-context learning (See Prompt Engineering) as utilizing short-term memory of the model to learn.\n * Long-term memory: This provides the agent with the capability to retain and recall (infinite) information over extended periods, often by leveraging an external vector store and fast retrieval.\n * **Tool use**\n * The agent learns to call external APIs for extra information that is missing from the model weights (often hard to change after pre-training), including current information, code execution c"


Nuclia automatically indexes your unstructured data from any internal and external source, providing optimized search results and generative answers. It can handle video and audio transcription, image content extraction, and document parsing.
The Nuclia Understanding API document transformer splits text into paragraphs and sentences, identifies entities, provides a summary of the text and generates embeddings for all the sentences.
To use the Nuclia Understanding API, you need to have a Nuclia account. You can create one for free at https://nuclia.cloud, and then create a NUA key.
from langchain.document_transformers.nuclia_text_transform import NucliaTextTransformer
#!pip install --upgrade protobuf
#!pip install nucliadb-protos
import os

os.environ["NUCLIA_ZONE"] = "<YOUR_ZONE>" # e.g. europe-1
os.environ["NUCLIA_NUA_KEY"] = "<YOUR_API_KEY>"
To use the Nuclia document transformer, you need to instantiate a NucliaUnderstandingAPI tool with enable_ml set to True:
from langchain.tools.nuclia import NucliaUnderstandingAPI

nua = NucliaUnderstandingAPI(enable_ml=True)
The Nuclia document transformer must be called in async mode, so you need to use the atransform_documents method:
import asyncio

from langchain.document_transformers.nuclia_text_transform import NucliaTextTransformer
from langchain.schema.document import Document

async def process():
documents = [
Document(page_content="<TEXT 1>", metadata={}),
Document(page_content="<TEXT 2>", metadata={}),
Document(page_content="<TEXT 3>", metadata={}),
]
nuclia_transformer = NucliaTextTransformer(nua)
transformed_documents = await nuclia_transformer.atransform_documents(documents)
print(transformed_documents)

asyncio.run(process())


OpenAI Functions Metadata Tagger
It can often be useful to tag ingested documents with structured metadata, such as the title, tone, or length of a document, to allow for more targeted similarity search later. However, for large numbers of documents, performing this labelling process manually can be tedious.
The OpenAIMetadataTagger document transformer automates this process by extracting metadata from each provided document according to a provided schema. It uses a configurable OpenAI Functions-powered chain under the hood, so if you pass a custom LLM instance, it must be an OpenAI model with functions support.
Note: This document transformer works best with complete documents, so it's best to run it first with whole documents before doing any other splitting or processing!
For example, let's say you wanted to index a set of movie reviews. You could initialize the document transformer with a valid JSON Schema object as follows:
from langchain.schema import Document
from langchain.chat_models import ChatOpenAI
from langchain.document_transformers.openai_functions import create_metadata_tagger
schema = {
"properties": {
"movie_title": {"type": "string"},
"critic": {"type": "string"},
"tone": {"type": "string", "enum": ["positive", "negative"]},
"rating": {
"type": "integer",
"description": "The number of stars the critic rated the movie",
},
},
"required": ["movie_title", "critic", "tone"],
}

# Must be an OpenAI model that supports functions
llm = ChatOpenAI(temperature=0, model="gpt-3.5-turbo-0613")

document_transformer = create_metadata_tagger(metadata_schema=schema, llm=llm)
You can then simply pass the document transformer a list of documents, and it will extract metadata from the contents:
original_documents = [
Document(
page_content="Review of The Bee Movie\nBy Roger Ebert\n\nThis is the greatest movie ever made. 4 out of 5 stars."
),
Document(
page_content="Review of The Godfather\nBy Anonymous\n\nThis movie was super boring. 1 out of 5 stars.",
metadata={"reliable": False},
),
]

enhanced_documents = document_transformer.transform_documents(original_documents)
import json


enhanced_documents = document_transformer.transform_documents(original_documents)
import json

print(
*[d.page_content + "\n\n" + json.dumps(d.metadata) for d in enhanced_documents],
sep="\n\n---------------\n\n"
)
Review of The Bee Movie
By Roger Ebert

This is the greatest movie ever made. 4 out of 5 stars.

{"movie_title": "The Bee Movie", "critic": "Roger Ebert", "tone": "positive", "rating": 4}

---------------

Review of The Godfather
By Anonymous

This movie was super boring. 1 out of 5 stars.

{"movie_title": "The Godfather", "critic": "Anonymous", "tone": "negative", "rating": 1, "reliable": false}
The new documents can then be further processed by a text splitter before being loaded into a vector store. Extracted fields will not overwrite existing metadata.
You can also initialize the document transformer with a Pydantic schema:
from typing import Literal

from pydantic import BaseModel, Field

class Properties(BaseModel):
movie_title: str
critic: str
tone: Literal["positive", "negative"]
rating: int = Field(description="Rating out of 5 stars")

document_transformer = create_metadata_tagger(Properties, llm)
enhanced_documents = document_transformer.transform_documents(original_documents)

print(
*[d.page_content + "\n\n" + json.dumps(d.metadata) for d in enhanced_documents],
sep="\n\n---------------\n\n"
)
Review of The Bee Movie
By Roger Ebert

This is the greatest movie ever made. 4 out of 5 stars.

{"movie_title": "The Bee Movie", "critic": "Roger Ebert", "tone": "positive", "rating": 4}

---------------

Review of The Godfather
By Anonymous

This movie was super boring. 1 out of 5 stars.


This movie was super boring. 1 out of 5 stars.

{"movie_title": "The Godfather", "critic": "Anonymous", "tone": "negative", "rating": 1, "reliable": false}
Customization
You can pass the underlying tagging chain the standard LLMChain arguments in the document transformer constructor. For example, if you wanted to ask the LLM to focus specific details in the input documents, or extract metadata in a certain style, you could pass in a custom prompt:
from langchain.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate.from_template(
"""Extract relevant information from the following text.
Anonymous critics are actually Roger Ebert.

{input}
"""
)

document_transformer = create_metadata_tagger(schema, llm, prompt=prompt)
enhanced_documents = document_transformer.transform_documents(original_documents)

print(
*[d.page_content + "\n\n" + json.dumps(d.metadata) for d in enhanced_documents],
sep="\n\n---------------\n\n"
)
Review of The Bee Movie
By Roger Ebert

This is the greatest movie ever made. 4 out of 5 stars.

{"movie_title": "The Bee Movie", "critic": "Roger Ebert", "tone": "positive", "rating": 4}

---------------

Review of The Godfather
By Anonymous

This movie was super boring. 1 out of 5 stars.

{"movie_title": "The Godfather", "critic": "Roger Ebert", "tone": "negative", "rating": 1, "reliable": false}