This notebook goes over how to use the google search component.
Then we will need to set some environment variables.
"STATE OF HAWAII. 1 Child's First Name. (Type or print). 2. Sex. BARACK. 3. This Birth. CERTIFICATE OF LIVE BIRTH. FILE. NUMBER 151 le. lb. Middle Name. Barack Hussein Obama II is an American former politician who served as the 44th president of the United States from 2009 to 2017. A member of the Democratic\xa0... When Barack Obama was elected president in 2008, he became the first African American to hold ... The Middle East remained a key foreign policy challenge. Jan 19, 2017 ... Jordan Barack Treasure, New York City, born in 2008 ... Jordan Barack Treasure made national news when he was the focus of a New York newspaper\xa0... Portrait of George Washington, the 1st President of the United States ... Portrait of Barack Obama, the 44th President of the United States\xa0... His full name is Barack Hussein Obama II. Since the “II” is simply because he was named for his father, his last name is Obama. Mar 22, 2008 ... Barry Obama decided that he didn't like his nickname. A few of his friends at Occidental College had already begun to call him Barack (his\xa0... Aug 18, 2017 ... It took him several seconds and multiple clues to remember former President Barack Obama's first name. Miller knew that every answer had to\xa0... Feb 9, 2015 ... Michael Jordan misspelled Barack Obama's first name on 50th-birthday gift ... Knowing Obama is a Chicagoan and huge basketball fan,\xa0... 4 days ago ... Barack Obama, in full Barack Hussein Obama II, (born August 4, 1961, Honolulu, Hawaii, U.S.), 44th president of the United States (2009–17) and\xa0..."
Run query through GoogleSearch and return snippet, title, and link metadata.


Golden provides a set of natural language APIs for querying and enrichment using the Golden Knowledge Graph e.g. queries such as: Products from OpenAI, Generative ai companies with series a funding, and rappers who invest can be used to retrieve structured data about relevant entities.
The golden-query langchain tool is a wrapper on top of the Golden Query API which enables programmatic access to these results. See the Golden Query API docs for more information.
This notebook goes over how to use the golden-query tool.
{'results': [{'id': 4673886,
'latestVersionId': 60276991,
'properties': [{'predicateId': 'name',
'instances': [{'value': 'Samsung', 'citations': []}]}]},
{'id': 7008,
'latestVersionId': 61087416,
'properties': [{'predicateId': 'name',
'instances': [{'value': 'Intel', 'citations': []}]}]},
{'id': 24193,
'latestVersionId': 60274482,
'properties': [{'predicateId': 'name',
'instances': [{'value': 'Texas Instruments', 'citations': []}]}]},
{'id': 1142,
'latestVersionId': 61406205,
'properties': [{'predicateId': 'name',
'instances': [{'value': 'Advanced Micro Devices', 'citations': []}]}]},
{'id': 193948,
'latestVersionId': 58326582,
'properties': [{'predicateId': 'name',
'instances': [{'value': 'Freescale Semiconductor', 'citations': []}]}]},
{'id': 91316,
'latestVersionId': 60387380,
'properties': [{'predicateId': 'name',
'instances': [{'value': 'Agilent Technologies', 'citations': []}]}]},
{'id': 90014,
'latestVersionId': 60388078,
'properties': [{'predicateId': 'name',
'instances': [{'value': 'Novartis', 'citations': []}]}]},
{'id': 237458,
'latestVersionId': 61406160,
'properties': [{'predicateId': 'name',


'properties': [{'predicateId': 'name',
'instances': [{'value': 'Analog Devices', 'citations': []}]}]},
{'id': 3941943,
'latestVersionId': 60382250,
'properties': [{'predicateId': 'name',
'instances': [{'value': 'AbbVie Inc.', 'citations': []}]}]},
{'id': 4178762,
'latestVersionId': 60542667,
'properties': [{'predicateId': 'name',
'instances': [{'value': 'IBM', 'citations': []}]}]}],
'next': 'https://golden.com/api/v2/public/queries/59044/results/?cursor=eyJwb3NpdGlvbiI6IFsxNzYxNiwgIklCTS04M1lQM1oiXX0%3D&pageSize=10',
'previous': None}


Google Drive
Google Drive is a file storage and synchronization service developed by Google.
This notebook covers how to load documents from Google Drive. Currently, only Google Docs are supported.
Prerequisites
Create a Google Cloud project or use an existing project
Enable the Google Drive API
Authorize credentials for desktop app
pip install --upgrade google-api-python-client google-auth-httplib2 google-auth-oauthlib
🧑 Instructions for ingesting your Google Docs data
By default, the GoogleDriveLoader expects the credentials.json file to be ~/.credentials/credentials.json, but this is configurable using the credentials_path keyword argument. Same thing with token.json - token_path. Note that token.json will be created automatically the first time you use the loader.
GoogleDriveLoader can load from a list of Google Docs document ids or a folder id. You can obtain your folder and document id from the URL:
Folder: https://drive.google.com/drive/u/0/folders/1yucgL9WGgWZdM1TOuKkeghlPizuzMYb5 -> folder id is "1yucgL9WGgWZdM1TOuKkeghlPizuzMYb5"
Document: https://docs.google.com/document/d/1bfaMQ18_i56204VaQDVeAFpqEijJTgvurupdEDiaUQw/edit -> document id is "1bfaMQ18_i56204VaQDVeAFpqEijJTgvurupdEDiaUQw"
pip install --upgrade google-api-python-client google-auth-httplib2 google-auth-oauthlib
from langchain.document_loaders import GoogleDriveLoader
loader = GoogleDriveLoader(
folder_id="1yucgL9WGgWZdM1TOuKkeghlPizuzMYb5",
# Optional: configure whether to recursively fetch files from subfolders. Defaults to False.
recursive=False,
)
When you pass a folder_id by default all files of type document, sheet and pdf are loaded. You can modify this behaviour by passing a file_types argument
loader = GoogleDriveLoader(
folder_id="1yucgL9WGgWZdM1TOuKkeghlPizuzMYb5",
file_types=["document", "sheet"],
recursive=False
)


file_types=["document", "sheet"],
recursive=False
)
Passing in Optional File Loaders
When processing files other than Google Docs and Google Sheets, it can be helpful to pass an optional file loader to GoogleDriveLoader. If you pass in a file loader, that file loader will be used on documents that do not have a Google Docs or Google Sheets MIME type. Here is an example of how to load an Excel document from Google Drive using a file loader.
from langchain.document_loaders import GoogleDriveLoader
from langchain.document_loaders import UnstructuredFileIOLoader
file_id = "1x9WBtFPWMEAdjcJzPScRsjpjQvpSo_kz"
loader = GoogleDriveLoader(
file_ids=[file_id],
file_loader_cls=UnstructuredFileIOLoader,
file_loader_kwargs={"mode": "elements"},
)
You can also process a folder with a mix of files and Google Docs/Sheets using the following pattern:
folder_id = "1asMOHY1BqBS84JcRbOag5LOJac74gpmD"
loader = GoogleDriveLoader(
folder_id=folder_id,
file_loader_cls=UnstructuredFileIOLoader,
file_loader_kwargs={"mode": "elements"},
)
Extended usage
An external component can manage the complexity of Google Drive : langchain-googledrive It's compatible with the ̀langchain.document_loaders.GoogleDriveLoader and can be used in its place.
To be compatible with containers, the authentication uses an environment variable ̀GOOGLE_ACCOUNT_FILE` to credential file (for user or service).
pip install langchain-googledrive
folder_id='root'
#folder_id='1yucgL9WGgWZdM1TOuKkeghlPizuzMYb5'
# Use the advanced version.
from langchain_googledrive.document_loaders import GoogleDriveLoader
loader = GoogleDriveLoader(
folder_id=folder_id,
recursive=False,
num_results=2, # Maximum number of file to load
)
By default, all files with these mime-type can be converted to Document.
text/text
text/plain
text/html
text/csv
text/markdown
image/png
image/jpeg
application/epub+zip


text/csv
text/markdown
image/png
image/jpeg
application/epub+zip
application/pdf
application/rtf
application/vnd.google-apps.document (GDoc)
application/vnd.google-apps.presentation (GSlide)
application/vnd.google-apps.spreadsheet (GSheet)
application/vnd.google.colaboratory (Notebook colab)
application/vnd.openxmlformats-officedocument.presentationml.presentation (PPTX)
application/vnd.openxmlformats-officedocument.wordprocessingml.document (DOCX)
It's possible to update or customize this. See the documentation of GDriveLoader.
But, the corresponding packages must be installed.
for doc in loader.load():
print("---")
print(doc.page_content.strip()[:60]+"...")
Customize the search pattern
All parameter compatible with Google list() API can be set.
To specify the new pattern of the Google request, you can use a PromptTemplate(). The variables for the prompt can be set with kwargs in the constructor. Some pre-formated request are proposed (use {query}, {folder_id} and/or {mime_type}):
You can customize the criteria to select the files. A set of predefined filter are proposed: | template | description | | -------------------------------------- | --------------------------------------------------------------------- | | gdrive-all-in-folder | Return all compatible files from a folder_id | | gdrive-query | Search query in all drives | | gdrive-by-name | Search file with name query | | gdrive-query-in-folder | Search query in folder_id (and sub-folders if recursive=true) | | gdrive-mime-type | Search a specific mime_type | | gdrive-mime-type-in-folder | Search a specific mime_type in folder_id | | gdrive-query-with-mime-type | Search query with a specific mime_type | | gdrive-query-with-mime-type-and-folder | Search query with a specific mime_type and in folder_id |
loader = GoogleDriveLoader(
folder_id=folder_id,
recursive=False,
template="gdrive-query", # Default template to use
query="machine learning",
num_results=2, # Maximum number of file to load
supportsAllDrives=False, # GDrive `list()` parameter
)
for doc in loader.load():
print("---")
print(doc.page_content.strip()[:60]+"...")
You can customize your pattern.


print(doc.page_content.strip()[:60]+"...")
You can customize your pattern.
from langchain.prompts.prompt import PromptTemplate
loader = GoogleDriveLoader(
folder_id=folder_id,
recursive=False,
template=PromptTemplate(
input_variables=["query", "query_name"],
template="fullText contains '{query}' and name contains '{query_name}' and trashed=false",
), # Default template to use
query="machine learning",
query_name="ML",
num_results=2, # Maximum number of file to load
)
for doc in loader.load():
print("---")
print(doc.page_content.strip()[:60]+"...")
Modes for GSlide and GSheet
The parameter mode accepts different values:
"document": return the body of each document
"snippets": return the description of each file (set in metadata of Google Drive files).
The conversion can manage in Markdown format:
bullet
link
table
titles
The parameter gslide_mode accepts different values:
"single" : one document with <PAGE BREAK>
"slide" : one document by slide
"elements" : one document for each elements.
loader = GoogleDriveLoader(
template="gdrive-mime-type",
mime_type="application/vnd.google-apps.presentation", # Only GSlide files
gslide_mode="slide",
num_results=2, # Maximum number of file to load
)
for doc in loader.load():
print("---")
print(doc.page_content.strip()[:60]+"...")
The parameter gsheet_mode accepts different values:
"single": Generate one document by line
"elements" : one document with markdown array and <PAGE BREAK> tags.
loader = GoogleDriveLoader(
template="gdrive-mime-type",
mime_type="application/vnd.google-apps.spreadsheet", # Only GSheet files
gsheet_mode="elements",
num_results=2, # Maximum number of file to load
)
for doc in loader.load():
print("---")
print(doc.page_content.strip()[:60]+"...")
Advanced usage
All Google File have a 'description' in the metadata. This field can be used to memorize a summary of the document or others indexed tags (See method lazy_update_description_with_summary()).


If you use the mode="snippet", only the description will be used for the body. Else, the metadata['summary'] has the field.
Sometime, a specific filter can be used to extract some information from the filename, to select some files with specific criteria. You can use a filter.
Sometimes, many documents are returned. It's not necessary to have all documents in memory at the same time. You can use the lazy versions of methods, to get one document at a time. It's better to use a complex query in place of a recursive search. For each folder, a query must be applied if you activate recursive=True.
import os
loader = GoogleDriveLoader(
gdrive_api_file=os.environ["GOOGLE_ACCOUNT_FILE"],
num_results=2,
template="gdrive-query",
filter=lambda search, file: "#test" not in file.get('description',''),
query='machine learning',
supportsAllDrives=False,
)
for doc in loader.load():
print("---")
print(doc.page_content.strip()[:60]+"...")


Clarifai
Clarifai is an AI Platform that provides the full AI lifecycle ranging from data exploration, data labeling, model training, evaluation, and inference.
This example goes over how to use LangChain to interact with Clarifai models.
To use Clarifai, you must have an account and a Personal Access Token (PAT) key. Check here to get or create a PAT.
Dependencies
# Install required dependencies
pip install clarifai
Imports
Here we will be setting the personal access token. You can find your PAT under settings/security in your Clarifai account.
# Please login and get your API key from https://clarifai.com/settings/security
from getpass import getpass

CLARIFAI_PAT = getpass()
# Import the required modules
from langchain.llms import Clarifai
from langchain import PromptTemplate, LLMChain
Input
Create a prompt template to be used with the LLM Chain:
template = """Question: {question}

Answer: Let's think step by step."""

prompt = PromptTemplate(template=template, input_variables=["question"])
Setup
Setup the user id and app id where the model resides. You can find a list of public models on https://clarifai.com/explore/models
You will have to also initialize the model id and if needed, the model version id. Some models have many versions, you can choose the one appropriate for your task.
USER_ID = "openai"
APP_ID = "chat-completion"
MODEL_ID = "GPT-3_5-turbo"

# You can provide a specific model version as the model_version_id arg.
# MODEL_VERSION_ID = "MODEL_VERSION_ID"
# Initialize a Clarifai LLM
clarifai_llm = Clarifai(
pat=CLARIFAI_PAT, user_id=USER_ID, app_id=APP_ID, model_id=MODEL_ID
)
# Create LLM chain
llm_chain = LLMChain(prompt=prompt, llm=clarifai_llm)
Run Chain
question = "What NFL team won the Super Bowl in the year Justin Beiber was born?"


llm_chain.run(question)
'Justin Bieber was born on March 1, 1994. So, we need to figure out the Super Bowl winner for the 1994 season. The NFL season spans two calendar years, so the Super Bowl for the 1994 season would have taken place in early 1995. \n\nThe Super Bowl in question is Super Bowl XXIX, which was played on January 29, 1995. The game was won by the San Francisco 49ers, who defeated the San Diego Chargers by a score of 49-26. Therefore, the San Francisco 49ers won the Super Bowl in the year Justin Bieber was born.'


Clarifai
Clarifai is an AI Platform that provides the full AI lifecycle ranging from data exploration, data labeling, model training, evaluation, and inference.
This example goes over how to use LangChain to interact with Clarifai models. Text embedding models in particular can be found here.
To use Clarifai, you must have an account and a Personal Access Token (PAT) key. Check here to get or create a PAT.
Dependencies
# Install required dependencies
pip install clarifai
Imports
Here we will be setting the personal access token. You can find your PAT under settings/security in your Clarifai account.
# Please login and get your API key from https://clarifai.com/settings/security
from getpass import getpass

CLARIFAI_PAT = getpass()
# Import the required modules
from langchain.embeddings import ClarifaiEmbeddings
from langchain import PromptTemplate, LLMChain
Input
Create a prompt template to be used with the LLM Chain:
template = """Question: {question}

Answer: Let's think step by step."""

prompt = PromptTemplate(template=template, input_variables=["question"])
Setup
Set the user id and app id to the application in which the model resides. You can find a list of public models on https://clarifai.com/explore/models
You will have to also initialize the model id and if needed, the model version id. Some models have many versions, you can choose the one appropriate for your task.
USER_ID = "salesforce"
APP_ID = "blip"
MODEL_ID = "multimodal-embedder-blip-2"

# You can provide a specific model version as the model_version_id arg.
# MODEL_VERSION_ID = "MODEL_VERSION_ID"
# Initialize a Clarifai embedding model
embeddings = ClarifaiEmbeddings(
pat=CLARIFAI_PAT, user_id=USER_ID, app_id=APP_ID, model_id=MODEL_ID
)
text = "This is a test document."
query_result = embeddings.embed_query(text)
doc_result = embeddings.embed_documents([text])


Chroma
Chroma is a AI-native open-source vector database focused on developer productivity and happiness. Chroma is licensed under Apache 2.0.
Install Chroma with:
Chroma runs in various modes. See below for examples of each integrated with LangChain.
in-memory - in a python script or jupyter notebook
in-memory with persistance - in a script or notebook and save/load to disk
in a docker container - as a server running your local machine or in the cloud
Like any other database, you can:
.add
.get
.update
.upsert
.delete
.peek
and .query runs the similarity search.
View full docs at docs. To access these methods directly, you can do ._collection_.method()
Basic Example
In this basic example, we take the most recent State of the Union Address, split it into chunks, embed it using an open-source embedding model, load it into Chroma, and then query it.
# import
from langchain.embeddings.sentence_transformer import SentenceTransformerEmbeddings
from langchain.text_splitter import CharacterTextSplitter
from langchain.vectorstores import Chroma
from langchain.document_loaders import TextLoader

# load the document and split it into chunks
loader = TextLoader("../../../state_of_the_union.txt")
documents = loader.load()

# split it into chunks
text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)
docs = text_splitter.split_documents(documents)

# create the open-source embedding function
embedding_function = SentenceTransformerEmbeddings(model_name="all-MiniLM-L6-v2")

# load it into Chroma
db = Chroma.from_documents(docs, embedding_function)

# query it
query = "What did the president say about Ketanji Brown Jackson"
docs = db.similarity_search(query)

# print results
print(docs[0].page_content)
/Users/jeff/.pyenv/versions/3.10.10/lib/python3.10/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
from .autonotebook import tqdm as notebook_tqdm


Tonight. I call on the Senate to: Pass the Freedom to Vote Act. Pass the John Lewis Voting Rights Act. And while you’re at it, pass the Disclose Act so Americans can know who is funding our elections.

Tonight, I’d like to honor someone who has dedicated his life to serve this country: Justice Stephen Breyer—an Army veteran, Constitutional scholar, and retiring Justice of the United States Supreme Court. Justice Breyer, thank you for your service.

One of the most serious constitutional responsibilities a President has is nominating someone to serve on the United States Supreme Court.

And I did that 4 days ago, when I nominated Circuit Court of Appeals Judge Ketanji Brown Jackson. One of our nation’s top legal minds, who will continue Justice Breyer’s legacy of excellence.
Basic Example (including saving to disk)
Extending the previous example, if you want to save to disk, simply initialize the Chroma client and pass the directory where you want the data to be saved to.
Caution: Chroma makes a best-effort to automatically save data to disk, however multiple in-memory clients can stomp each other's work. As a best practice, only have one client per path running at any given time.
# save to disk
db2 = Chroma.from_documents(docs, embedding_function, persist_directory="./chroma_db")
docs = db2.similarity_search(query)

# load from disk
db3 = Chroma(persist_directory="./chroma_db", embedding_function=embedding_function)
docs = db3.similarity_search(query)
print(docs[0].page_content)
Tonight. I call on the Senate to: Pass the Freedom to Vote Act. Pass the John Lewis Voting Rights Act. And while you’re at it, pass the Disclose Act so Americans can know who is funding our elections.

Tonight, I’d like to honor someone who has dedicated his life to serve this country: Justice Stephen Breyer—an Army veteran, Constitutional scholar, and retiring Justice of the United States Supreme Court. Justice Breyer, thank you for your service.

One of the most serious constitutional responsibilities a President has is nominating someone to serve on the United States Supreme Court.


And I did that 4 days ago, when I nominated Circuit Court of Appeals Judge Ketanji Brown Jackson. One of our nation’s top legal minds, who will continue Justice Breyer’s legacy of excellence.
Passing a Chroma Client into Langchain
You can also create a Chroma Client and pass it to LangChain. This is particularly useful if you want easier access to the underlying database.
You can also specify the collection name that you want LangChain to use.
import chromadb

persistent_client = chromadb.PersistentClient()
collection = persistent_client.get_or_create_collection("collection_name")
collection.add(ids=["1", "2", "3"], documents=["a", "b", "c"])

langchain_chroma = Chroma(
client=persistent_client,
collection_name="collection_name",
embedding_function=embedding_function,
)

print("There are", langchain_chroma._collection.count(), "in the collection")
Add of existing embedding ID: 1
Add of existing embedding ID: 2
Add of existing embedding ID: 3
Add of existing embedding ID: 1
Add of existing embedding ID: 2
Add of existing embedding ID: 3
Add of existing embedding ID: 1
Insert of existing embedding ID: 1
Add of existing embedding ID: 2
Insert of existing embedding ID: 2
Add of existing embedding ID: 3
Insert of existing embedding ID: 3


There are 3 in the collection
Basic Example (using the Docker Container)
You can also run the Chroma Server in a Docker container separately, create a Client to connect to it, and then pass that to LangChain.
Chroma has the ability to handle multiple Collections of documents, but the LangChain interface expects one, so we need to specify the collection name. The default collection name used by LangChain is "langchain".
Here is how to clone, build, and run the Docker Image:
git clone git@github.com:chroma-core/chroma.git
Edit the docker-compose.yml file and add ALLOW_RESET=TRUE under environment
...
command: uvicorn chromadb.app:app --reload --workers 1 --host 0.0.0.0 --port 8000 --log-config log_config.yml
environment:
- IS_PERSISTENT=TRUE
- ALLOW_RESET=TRUE
ports:
- 8000:8000
...
Then run docker-compose up -d --build
# create the chroma client
import chromadb
import uuid
from chromadb.config import Settings

client = chromadb.HttpClient(settings=Settings(allow_reset=True))
client.reset() # resets the database
collection = client.create_collection("my_collection")
for doc in docs:
collection.add(
ids=[str(uuid.uuid1())], metadatas=doc.metadata, documents=doc.page_content
)

# tell LangChain to use our client and collection name
db4 = Chroma(client=client, collection_name="my_collection", embedding_function=embedding_function)
query = "What did the president say about Ketanji Brown Jackson"
docs = db4.similarity_search(query)
print(docs[0].page_content)
Tonight. I call on the Senate to: Pass the Freedom to Vote Act. Pass the John Lewis Voting Rights Act. And while you’re at it, pass the Disclose Act so Americans can know who is funding our elections.

Tonight, I’d like to honor someone who has dedicated his life to serve this country: Justice Stephen Breyer—an Army veteran, Constitutional scholar, and retiring Justice of the United States Supreme Court. Justice Breyer, thank you for your service.

One of the most serious constitutional responsibilities a President has is nominating someone to serve on the United States Supreme Court.


And I did that 4 days ago, when I nominated Circuit Court of Appeals Judge Ketanji Brown Jackson. One of our nation’s top legal minds, who will continue Justice Breyer’s legacy of excellence.
Update and Delete
While building toward a real application, you want to go beyond adding data, and also update and delete data.
Chroma has users provide ids to simplify the bookkeeping here. ids can be the name of the file, or a combined has like filename_paragraphNumber, etc.
Chroma supports all these operations - though some of them are still being integrated all the way through the LangChain interface. Additional workflow improvements will be added soon.
Here is a basic example showing how to do various operations:
# create simple ids
ids = [str(i) for i in range(1, len(docs) + 1)]

# add data
example_db = Chroma.from_documents(docs, embedding_function, ids=ids)
docs = example_db.similarity_search(query)
print(docs[0].metadata)

# update the metadata for a document
docs[0].metadata = {
"source": "../../../state_of_the_union.txt",
"new_value": "hello world",
}
example_db.update_document(ids[0], docs[0])
print(example_db._collection.get(ids=[ids[0]]))


# delete the last document
print("count before", example_db._collection.count())
example_db._collection.delete(ids=[ids[-1]])
print("count after", example_db._collection.count())
{'source': '../../../state_of_the_union.txt'}
{'ids': ['1'], 'embeddings': None, 'metadatas': [{'new_value': 'hello world', 'source': '../../../state_of_the_union.txt'}], 'documents': ['Tonight. I call on the Senate to: Pass the Freedom to Vote Act. Pass the John Lewis Voting Rights Act. And while you’re at it, pass the Disclose Act so Americans can know who is funding our elections. \n\nTonight, I’d like to honor someone who has dedicated his life to serve this country: Justice Stephen Breyer—an Army veteran, Constitutional scholar, and retiring Justice of the United States Supreme Court. Justice Breyer, thank you for your service. \n\nOne of the most serious constitutional responsibilities a President has is nominating someone to serve on the United States Supreme Court. \n\nAnd I did that 4 days ago, when I nominated Circuit Court of Appeals Judge Ketanji Brown Jackson. One of our nation’s top legal minds, who will continue Justice Breyer’s legacy of excellence.']}
count before 46
count after 45
Use OpenAI Embeddings
Many people like to use OpenAIEmbeddings, here is how to set that up.
# get a token: https://platform.openai.com/account/api-keys

from getpass import getpass
from langchain.embeddings.openai import OpenAIEmbeddings

OPENAI_API_KEY = getpass()
import os

os.environ["OPENAI_API_KEY"] = OPENAI_API_KEY
embeddings = OpenAIEmbeddings()
new_client = chromadb.EphemeralClient()
openai_lc_client = Chroma.from_documents(
docs, embeddings, client=new_client, collection_name="openai_collection"
)

query = "What did the president say about Ketanji Brown Jackson"
docs = openai_lc_client.similarity_search(query)
print(docs[0].page_content)
Tonight. I call on the Senate to: Pass the Freedom to Vote Act. Pass the John Lewis Voting Rights Act. And while you’re at it, pass the Disclose Act so Americans can know who is funding our elections.


Tonight, I’d like to honor someone who has dedicated his life to serve this country: Justice Stephen Breyer—an Army veteran, Constitutional scholar, and retiring Justice of the United States Supreme Court. Justice Breyer, thank you for your service.

One of the most serious constitutional responsibilities a President has is nominating someone to serve on the United States Supreme Court.


And I did that 4 days ago, when I nominated Circuit Court of Appeals Judge Ketanji Brown Jackson. One of our nation’s top legal minds, who will continue Justice Breyer’s legacy of excellence.
Other Information
Similarity search with score
The returned distance score is cosine distance. Therefore, a lower score is better.
docs = db.similarity_search_with_score(query)
(Document(page_content='Tonight. I call on the Senate to: Pass the Freedom to Vote Act. Pass the John Lewis Voting Rights Act. And while you’re at it, pass the Disclose Act so Americans can know who is funding our elections. \n\nTonight, I’d like to honor someone who has dedicated his life to serve this country: Justice Stephen Breyer—an Army veteran, Constitutional scholar, and retiring Justice of the United States Supreme Court. Justice Breyer, thank you for your service. \n\nOne of the most serious constitutional responsibilities a President has is nominating someone to serve on the United States Supreme Court. \n\nAnd I did that 4 days ago, when I nominated Circuit Court of Appeals Judge Ketanji Brown Jackson. One of our nation’s top legal minds, who will continue Justice Breyer’s legacy of excellence.', metadata={'source': '../../../state_of_the_union.txt'}),
1.1972057819366455)
Retriever options
This section goes over different options for how to use Chroma as a retriever.
MMR
In addition to using similarity search in the retriever object, you can also use mmr.
retriever = db.as_retriever(search_type="mmr")
retriever.get_relevant_documents(query)[0]


retriever.get_relevant_documents(query)[0]
Document(page_content='Tonight. I call on the Senate to: Pass the Freedom to Vote Act. Pass the John Lewis Voting Rights Act. And while you’re at it, pass the Disclose Act so Americans can know who is funding our elections. \n\nTonight, I’d like to honor someone who has dedicated his life to serve this country: Justice Stephen Breyer—an Army veteran, Constitutional scholar, and retiring Justice of the United States Supreme Court. Justice Breyer, thank you for your service. \n\nOne of the most serious constitutional responsibilities a President has is nominating someone to serve on the United States Supreme Court. \n\nAnd I did that 4 days ago, when I nominated Circuit Court of Appeals Judge Ketanji Brown Jackson. One of our nation’s top legal minds, who will continue Justice Breyer’s legacy of excellence.', metadata={'source': '../../../state_of_the_union.txt'})
Filtering on metadata
It can be helpful to narrow down the collection before working with it.
For example, collections can be filtered on metadata using the get method.
# filter collection for updated source
example_db.get(where={"source": "some_other_source"})
{'ids': [], 'embeddings': None, 'metadatas': [], 'documents': []}


C Transformers
The C Transformers library provides Python bindings for GGML models.
This example goes over how to use LangChain to interact with C Transformers models.
Install
%pip install ctransformers
Load Model
from langchain.llms import CTransformers

llm = CTransformers(model="marella/gpt-2-ggml")
Generate Text
print(llm("AI is going to"))
Streaming
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler

llm = CTransformers(
model="marella/gpt-2-ggml", callbacks=[StreamingStdOutCallbackHandler()]
)

response = llm("AI is going to")
LLMChain
from langchain import PromptTemplate, LLMChain

template = """Question: {question}

Answer:"""

prompt = PromptTemplate(template=template, input_variables=["question"])

llm_chain = LLMChain(prompt=prompt, llm=llm)

response = llm_chain.run("What is AI?")


Cohere
Let's load the Cohere Embedding class.
from langchain.embeddings import CohereEmbeddings
embeddings = CohereEmbeddings(cohere_api_key=cohere_api_key)
text = "This is a test document."
query_result = embeddings.embed_query(text)
doc_result = embeddings.embed_documents([text])


DashVector
DashVector is a fully-managed vectorDB service that supports high-dimension dense and sparse vectors, real-time insertion and filtered search. It is built to scale automatically and can adapt to different application requirements.
This notebook shows how to use functionality related to the DashVector vector database.
To use DashVector, you must have an API key. Here are the installation instructions.
Install
pip install dashvector dashscope
We want to use DashScopeEmbeddings so we also have to get the Dashscope API Key.
import os
import getpass

os.environ["DASHVECTOR_API_KEY"] = getpass.getpass("DashVector API Key:")
os.environ["DASHSCOPE_API_KEY"] = getpass.getpass("DashScope API Key:")
Example
from langchain.embeddings.dashscope import DashScopeEmbeddings
from langchain.text_splitter import CharacterTextSplitter
from langchain.vectorstores import DashVector
from langchain.document_loaders import TextLoader

loader = TextLoader("../../modules/state_of_the_union.txt")
documents = loader.load()
text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)
docs = text_splitter.split_documents(documents)

embeddings = DashScopeEmbeddings()
We can create DashVector from documents.
dashvector = DashVector.from_documents(docs, embeddings)

query = "What did the president say about Ketanji Brown Jackson"
docs = dashvector.similarity_search(query)
print(docs)
We can add texts with meta datas and ids, and search with meta filter.
texts = ["foo", "bar", "baz"]
metadatas = [{"key": i} for i in range(len(texts))]
ids = ["0", "1", "2"]

dashvector.add_texts(texts, metadatas=metadatas, ids=ids)

docs = dashvector.similarity_search("foo", filter="key = 2")
print(docs)
[Document(page_content='baz', metadata={'key': 2})]


SQL Database
This notebook showcases an agent designed to interact with a SQL databases. The agent builds off of SQLDatabaseChain and is designed to answer more general questions about a database, as well as recover from errors.
Note that, as this agent is in active development, all answers might not be correct. Additionally, it is not guaranteed that the agent won't perform DML statements on your database given certain questions. Be careful running it on sensitive data!
This uses the example Chinook database. To set it up follow the instructions on https://database.guide/2-sample-databases-sqlite/, placing the .db file in a notebooks folder at the root of this repository.
Initialization
from langchain.agents import create_sql_agent
from langchain.agents.agent_toolkits import SQLDatabaseToolkit
from langchain.sql_database import SQLDatabase
from langchain.llms.openai import OpenAI
from langchain.agents import AgentExecutor
from langchain.agents.agent_types import AgentType
from langchain.chat_models import ChatOpenAI
db = SQLDatabase.from_uri("sqlite:///../../../../../notebooks/Chinook.db")
toolkit = SQLDatabaseToolkit(db=db, llm=OpenAI(temperature=0))
Using ZERO_SHOT_REACT_DESCRIPTION
This shows how to initialize the agent using the ZERO_SHOT_REACT_DESCRIPTION agent type.
agent_executor = create_sql_agent(
llm=OpenAI(temperature=0),
toolkit=toolkit,
verbose=True,
agent_type=AgentType.ZERO_SHOT_REACT_DESCRIPTION,
)
Using OpenAI Functions
This shows how to initialize the agent using the OPENAI_FUNCTIONS agent type. Note that this is an alternative to the above.
# agent_executor = create_sql_agent(
# llm=ChatOpenAI(temperature=0, model="gpt-3.5-turbo-0613"),
# toolkit=toolkit,
# verbose=True,
# agent_type=AgentType.OPENAI_FUNCTIONS
# )
Disclamer ⚠
The query chain may generate insert/update/delete queries. When this is not expected, use a custom prompt or create a SQL users without write permissions.
The final user might overload your SQL database by asking a simple question such as "run the biggest query possible". The generated query might look like:
SELECT * FROM "public"."users"


SELECT * FROM "public"."users"
JOIN "public"."user_permissions" ON "public"."users".id = "public"."user_permissions".user_id
JOIN "public"."projects" ON "public"."users".id = "public"."projects".user_id
JOIN "public"."events" ON "public"."projects".id = "public"."events".project_id;
For a transactional SQL database, if one of the table above contains millions of rows, the query might cause trouble to other applications using the same database.
Most datawarehouse oriented databases support user-level quota, for limiting resource usage.
Example: describing a table
agent_executor.run("Describe the playlisttrack table")


> Entering new chain...

Invoking: `list_tables_sql_db` with `{}`

Album, Artist, Track, PlaylistTrack, InvoiceLine, sales_table, Playlist, Genre, Employee, Customer, Invoice, MediaType
Invoking: `schema_sql_db` with `PlaylistTrack`

CREATE TABLE "PlaylistTrack" (
"PlaylistId" INTEGER NOT NULL,
"TrackId" INTEGER NOT NULL,
PRIMARY KEY ("PlaylistId", "TrackId"),
FOREIGN KEY("TrackId") REFERENCES "Track" ("TrackId"),
FOREIGN KEY("PlaylistId") REFERENCES "Playlist" ("PlaylistId")
)

/*
3 rows from PlaylistTrack table:
PlaylistId TrackId
1 3402
1 3389
1 3390
*/The `PlaylistTrack` table has two columns: `PlaylistId` and `TrackId`. It is a junction table that represents the relationship between playlists and tracks.

Here is the schema of the `PlaylistTrack` table:

```
CREATE TABLE "PlaylistTrack" (
"PlaylistId" INTEGER NOT NULL,
"TrackId" INTEGER NOT NULL,
PRIMARY KEY ("PlaylistId", "TrackId"),
FOREIGN KEY("TrackId") REFERENCES "Track" ("TrackId"),
FOREIGN KEY("PlaylistId") REFERENCES "Playlist" ("PlaylistId")
)
```

Here are three sample rows from the `PlaylistTrack` table:

```
PlaylistId TrackId
1 3402
1 3389
1 3390
```

Please let me know if there is anything else I can help you with.

> Finished chain.


> Finished chain.

'The `PlaylistTrack` table has two columns: `PlaylistId` and `TrackId`. It is a junction table that represents the relationship between playlists and tracks. \n\nHere is the schema of the `PlaylistTrack` table:\n\n```\nCREATE TABLE "PlaylistTrack" (\n\t"PlaylistId" INTEGER NOT NULL, \n\t"TrackId" INTEGER NOT NULL, \n\tPRIMARY KEY ("PlaylistId", "TrackId"), \n\tFOREIGN KEY("TrackId") REFERENCES "Track" ("TrackId"), \n\tFOREIGN KEY("PlaylistId") REFERENCES "Playlist" ("PlaylistId")\n)\n```\n\nHere are three sample rows from the `PlaylistTrack` table:\n\n```\nPlaylistId TrackId\n1 3402\n1 3389\n1 3390\n```\n\nPlease let me know if there is anything else I can help you with.'
Example: describing a table, recovering from an error
In this example, the agent tries to search for a table that doesn't exist, but finds the next best result
agent_executor.run("Describe the playlistsong table")

> Entering new AgentExecutor chain...
Action: list_tables_sql_db
Action Input: ""
Observation: Genre, PlaylistTrack, MediaType, Invoice, InvoiceLine, Track, Playlist, Customer, Album, Employee, Artist
Thought: I should look at the schema of the PlaylistSong table
Action: schema_sql_db
Action Input: "PlaylistSong"
Observation: Error: table_names {'PlaylistSong'} not found in database
Thought: I should check the spelling of the table
Action: list_tables_sql_db
Action Input: ""
Observation: Genre, PlaylistTrack, MediaType, Invoice, InvoiceLine, Track, Playlist, Customer, Album, Employee, Artist
Thought: The table is called PlaylistTrack
Action: schema_sql_db
Action Input: "PlaylistTrack"
Observation:
CREATE TABLE "PlaylistTrack" (
"PlaylistId" INTEGER NOT NULL,
"TrackId" INTEGER NOT NULL,
PRIMARY KEY ("PlaylistId", "TrackId"),
FOREIGN KEY("TrackId") REFERENCES "Track" ("TrackId"),
FOREIGN KEY("PlaylistId") REFERENCES "Playlist" ("PlaylistId")
)


SELECT * FROM 'PlaylistTrack' LIMIT 3;
PlaylistId TrackId
1 3402
1 3389
1 3390
Thought: I now know the final answer
Final Answer: The PlaylistTrack table contains two columns, PlaylistId and TrackId, which are both integers and are used to link Playlist and Track tables.

> Finished chain.

'The PlaylistTrack table contains two columns, PlaylistId and TrackId, which are both integers and are used to link Playlist and Track tables.'
Example: running queries
agent_executor.run(
"List the total sales per country. Which country's customers spent the most?"
)

> Entering new AgentExecutor chain...
Action: list_tables_sql_db
Action Input: ""
Observation: Invoice, MediaType, Artist, InvoiceLine, Genre, Playlist, Employee, Album, PlaylistTrack, Track, Customer
Thought: I should look at the schema of the relevant tables to see what columns I can use.
Action: schema_sql_db
Action Input: "Invoice, Customer"
Observation:
CREATE TABLE "Customer" (
"CustomerId" INTEGER NOT NULL,
"FirstName" NVARCHAR(40) NOT NULL,
"LastName" NVARCHAR(20) NOT NULL,
"Company" NVARCHAR(80),
"Address" NVARCHAR(70),
"City" NVARCHAR(40),
"State" NVARCHAR(40),
"Country" NVARCHAR(40),
"PostalCode" NVARCHAR(10),
"Phone" NVARCHAR(24),
"Fax" NVARCHAR(24),
"Email" NVARCHAR(60) NOT NULL,
"SupportRepId" INTEGER,
PRIMARY KEY ("CustomerId"),
FOREIGN KEY("SupportRepId") REFERENCES "Employee" ("EmployeeId")
)


SELECT * FROM 'Customer' LIMIT 3;
CustomerId FirstName LastName Company Address City State Country PostalCode Phone Fax Email SupportRepId
1 Luís Gonçalves Embraer - Empresa Brasileira de Aeronáutica S.A. Av. Brigadeiro Faria Lima, 2170 São José dos Campos SP Brazil 12227-000 +55 (12) 3923-5555 +55 (12) 3923-5566 luisg@embraer.com.br 3
2 Leonie Köhler None Theodor-Heuss-Straße 34 Stuttgart None Germany 70174 +49 0711 2842222 None leonekohler@surfeu.de 5
3 François Tremblay None 1498 rue Bélanger Montréal QC Canada H2G 1A7 +1 (514) 721-4711 None ftremblay@gmail.com 3

CREATE TABLE "Invoice" (
"InvoiceId" INTEGER NOT NULL,
"CustomerId" INTEGER NOT NULL,
"InvoiceDate" DATETIME NOT NULL,
"BillingAddress" NVARCHAR(70),
"BillingCity" NVARCHAR(40),
"BillingState" NVARCHAR(40),
"BillingCountry" NVARCHAR(40),
"BillingPostalCode" NVARCHAR(10),
"Total" NUMERIC(10, 2) NOT NULL,
PRIMARY KEY ("InvoiceId"),
FOREIGN KEY("CustomerId") REFERENCES "Customer" ("CustomerId")
)


SELECT * FROM 'Invoice' LIMIT 3;
InvoiceId CustomerId InvoiceDate BillingAddress BillingCity BillingState BillingCountry BillingPostalCode Total
1 2 2009-01-01 00:00:00 Theodor-Heuss-Straße 34 Stuttgart None Germany 70174 1.98
2 4 2009-01-02 00:00:00 Ullevålsveien 14 Oslo None Norway 0171 3.96
3 8 2009-01-03 00:00:00 Grétrystraat 63 Brussels None Belgium 1000 5.94
Thought: I should query the Invoice and Customer tables to get the total sales per country.
Action: query_sql_db
Action Input: SELECT c.Country, SUM(i.Total) AS TotalSales FROM Invoice i INNER JOIN Customer c ON i.CustomerId = c.CustomerId GROUP BY c.Country ORDER BY TotalSales DESC LIMIT 10
Observation: [('USA', 523.0600000000003), ('Canada', 303.9599999999999), ('France', 195.09999999999994), ('Brazil', 190.09999999999997), ('Germany', 156.48), ('United Kingdom', 112.85999999999999), ('Czech Republic', 90.24000000000001), ('Portugal', 77.23999999999998), ('India', 75.25999999999999), ('Chile', 46.62)]
Thought: I now know the final answer
Final Answer: The customers from the USA spent the most, with a total of $523.06.

> Finished chain.

'The customers from the USA spent the most, with a total of $523.06.'
agent_executor.run(
"Show the total number of tracks in each playlist. The Playlist name should be included in the result."
)


> Entering new AgentExecutor chain...
Action: list_tables_sql_db
Action Input: ""
Observation: Invoice, MediaType, Artist, InvoiceLine, Genre, Playlist, Employee, Album, PlaylistTrack, Track, Customer
Thought: I should look at the schema of the Playlist and PlaylistTrack tables to see what columns I can use.
Action: schema_sql_db
Action Input: "Playlist, PlaylistTrack"
Observation:
CREATE TABLE "Playlist" (
"PlaylistId" INTEGER NOT NULL,
"Name" NVARCHAR(120),
PRIMARY KEY ("PlaylistId")
)

SELECT * FROM 'Playlist' LIMIT 3;
PlaylistId Name
1 Music
2 Movies
3 TV Shows

CREATE TABLE "PlaylistTrack" (
"PlaylistId" INTEGER NOT NULL,
"TrackId" INTEGER NOT NULL,
PRIMARY KEY ("PlaylistId", "TrackId"),
FOREIGN KEY("TrackId") REFERENCES "Track" ("TrackId"),
FOREIGN KEY("PlaylistId") REFERENCES "Playlist" ("PlaylistId")
)

SELECT * FROM 'PlaylistTrack' LIMIT 3;
PlaylistId TrackId
1 3402
1 3389
1 3390
Thought: I can use a SELECT statement to get the total number of tracks in each playlist.
Action: query_checker_sql_db
Action Input: SELECT Playlist.Name, COUNT(PlaylistTrack.TrackId) AS TotalTracks FROM Playlist INNER JOIN PlaylistTrack ON Playlist.PlaylistId = PlaylistTrack.PlaylistId GROUP BY Playlist.Name
Observation:


SELECT Playlist.Name, COUNT(PlaylistTrack.TrackId) AS TotalTracks FROM Playlist INNER JOIN PlaylistTrack ON Playlist.PlaylistId = PlaylistTrack.PlaylistId GROUP BY Playlist.Name
Thought: The query looks correct, I can now execute it.
Action: query_sql_db
Action Input: SELECT Playlist.Name, COUNT(PlaylistTrack.TrackId) AS TotalTracks FROM Playlist INNER JOIN PlaylistTrack ON Playlist.PlaylistId = PlaylistTrack.PlaylistId GROUP BY Playlist.Name LIMIT 10
Observation: [('90’s Music', 1477), ('Brazilian Music', 39), ('Classical', 75), ('Classical 101 - Deep Cuts', 25), ('Classical 101 - Next Steps', 25), ('Classical 101 - The Basics', 25), ('Grunge', 15), ('Heavy Metal Classic', 26), ('Music', 6580), ('Music Videos', 1)]
Thought: I now know the final answer.
Final Answer: The total number of tracks in each playlist are: '90’s Music' (1477), 'Brazilian Music' (39), 'Classical' (75), 'Classical 101 - Deep Cuts' (25), 'Classical 101 - Next Steps' (25), 'Classical 101 - The Basics' (25), 'Grunge' (15), 'Heavy Metal Classic' (26), 'Music' (6580), 'Music Videos' (1).

> Finished chain.

"The total number of tracks in each playlist are: '90’s Music' (1477), 'Brazilian Music' (39), 'Classical' (75), 'Classical 101 - Deep Cuts' (25), 'Classical 101 - Next Steps' (25), 'Classical 101 - The Basics' (25), 'Grunge' (15), 'Heavy Metal Classic' (26), 'Music' (6580), 'Music Videos' (1)."
Recovering from an error
In this example, the agent is able to recover from an error after initially trying to access an attribute (Track.ArtistId) which doesn't exist.
agent_executor.run("Who are the top 3 best selling artists?")


> Entering new AgentExecutor chain...
Action: list_tables_sql_db
Action Input: ""
Observation: MediaType, Track, Invoice, Album, Playlist, Customer, Employee, InvoiceLine, PlaylistTrack, Genre, Artist
Thought: I should look at the schema of the Artist, InvoiceLine, and Track tables to see what columns I can use.
Action: schema_sql_db
Action Input: "Artist, InvoiceLine, Track"
Observation:
CREATE TABLE "Artist" (
"ArtistId" INTEGER NOT NULL,
"Name" NVARCHAR(120),
PRIMARY KEY ("ArtistId")
)

SELECT * FROM 'Artist' LIMIT 3;
ArtistId Name
1 AC/DC
2 Accept
3 Aerosmith

CREATE TABLE "Track" (
"TrackId" INTEGER NOT NULL,
"Name" NVARCHAR(200) NOT NULL,
"AlbumId" INTEGER,
"MediaTypeId" INTEGER NOT NULL,
"GenreId" INTEGER,
"Composer" NVARCHAR(220),
"Milliseconds" INTEGER NOT NULL,
"Bytes" INTEGER,
"UnitPrice" NUMERIC(10, 2) NOT NULL,
PRIMARY KEY ("TrackId"),
FOREIGN KEY("MediaTypeId") REFERENCES "MediaType" ("MediaTypeId"),
FOREIGN KEY("GenreId") REFERENCES "Genre" ("GenreId"),
FOREIGN KEY("AlbumId") REFERENCES "Album" ("AlbumId")
)

SELECT * FROM 'Track' LIMIT 3;
TrackId Name AlbumId MediaTypeId GenreId Composer Milliseconds Bytes UnitPrice
1 For Those About To Rock (We Salute You) 1 1 1 Angus Young, Malcolm Young, Brian Johnson 343719 11170334 0.99
2 Balls to the Wall 2 2 1 None 342562 5510424 0.99
3 Fast As a Shark 3 2 1 F. Baltes, S. Kaufman, U. Dirkscneider & W. Hoffman 230619 3990994 0.99


CREATE TABLE "InvoiceLine" (
"InvoiceLineId" INTEGER NOT NULL,
"InvoiceId" INTEGER NOT NULL,
"TrackId" INTEGER NOT NULL,
"UnitPrice" NUMERIC(10, 2) NOT NULL,
"Quantity" INTEGER NOT NULL,
PRIMARY KEY ("InvoiceLineId"),
FOREIGN KEY("TrackId") REFERENCES "Track" ("TrackId"),
FOREIGN KEY("InvoiceId") REFERENCES "Invoice" ("InvoiceId")
)

SELECT * FROM 'InvoiceLine' LIMIT 3;
InvoiceLineId InvoiceId TrackId UnitPrice Quantity
1 1 2 0.99 1
2 1 4 0.99 1
3 2 6 0.99 1
Thought: I should query the database to get the top 3 best selling artists.
Action: query_sql_db
Action Input: SELECT Artist.Name, SUM(InvoiceLine.Quantity) AS TotalQuantity FROM Artist INNER JOIN Track ON Artist.ArtistId = Track.ArtistId INNER JOIN InvoiceLine ON Track.TrackId = InvoiceLine.TrackId GROUP BY Artist.Name ORDER BY TotalQuantity DESC LIMIT 3
Observation: Error: (sqlite3.OperationalError) no such column: Track.ArtistId
[SQL: SELECT Artist.Name, SUM(InvoiceLine.Quantity) AS TotalQuantity FROM Artist INNER JOIN Track ON Artist.ArtistId = Track.ArtistId INNER JOIN InvoiceLine ON Track.TrackId = InvoiceLine.TrackId GROUP BY Artist.Name ORDER BY TotalQuantity DESC LIMIT 3]
(Background on this error at: https://sqlalche.me/e/14/e3q8)
Thought: I should double check my query before executing it.
Action: query_checker_sql_db
Action Input: SELECT Artist.Name, SUM(InvoiceLine.Quantity) AS TotalQuantity FROM Artist INNER JOIN Track ON Artist.ArtistId = Track.ArtistId INNER JOIN InvoiceLine ON Track.TrackId = InvoiceLine.TrackId GROUP BY Artist.Name ORDER BY TotalQuantity DESC LIMIT 3
Observation:


SELECT Artist.Name, SUM(InvoiceLine.Quantity) AS TotalQuantity
FROM Artist
INNER JOIN Track ON Artist.ArtistId = Track.ArtistId
INNER JOIN InvoiceLine ON Track.TrackId = InvoiceLine.TrackId
GROUP BY Artist.Name
ORDER BY TotalQuantity DESC
LIMIT 3;
Thought: I now know the final answer.
Action: query_sql_db
Action Input: SELECT Artist.Name, SUM(InvoiceLine.Quantity) AS TotalQuantity FROM Artist INNER JOIN Album ON Artist.ArtistId = Album.ArtistId INNER JOIN Track ON Album.AlbumId = Track.AlbumId INNER JOIN InvoiceLine ON Track.TrackId = InvoiceLine.TrackId GROUP BY Artist.Name ORDER BY TotalQuantity DESC LIMIT 3
Observation: [('Iron Maiden', 140), ('U2', 107), ('Metallica', 91)]
Thought: I now know the final answer.
Final Answer: The top 3 best selling artists are Iron Maiden, U2, and Metallica.

> Finished chain.

'The top 3 best selling artists are Iron Maiden, U2, and Metallica.'


This notebook goes over how to use the Google Serper component to search the web. First you need to sign up for a free account at serper.dev and get your api key.
from langchain.utilities import GoogleSerperAPIWrapper
from langchain.llms.openai import OpenAI
from langchain.agents import initialize_agent, Tool
from langchain.agents import AgentType

llm = OpenAI(temperature=0)
search = GoogleSerperAPIWrapper()
tools = [
Tool(
name="Intermediate Answer",
func=search.run,
description="useful for when you need to ask with search",
)
]


self_ask_with_search = initialize_agent(
tools, llm, agent=AgentType.SELF_ASK_WITH_SEARCH, verbose=True
)
self_ask_with_search.run(
"What is the hometown of the reigning men's U.S. Open champion?"
)
If you would also like to obtain the results in a structured way including metadata. For this we will be using the results method of the wrapper.
{'searchParameters': {'q': 'Apple Inc.',
'gl': 'us',
'hl': 'en',
'num': 10,
'type': 'search'},
'knowledgeGraph': {'title': 'Apple',
'type': 'Technology company',
'website': 'http://www.apple.com/',
'imageUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQwGQRv5TjjkycpctY66mOg_e2-npacrmjAb6_jAWhzlzkFE3OTjxyzbA&s=0',
'description': 'Apple Inc. is an American multinational '
'technology company headquartered in '
'Cupertino, California. Apple is the '
"world's largest technology company by "
'revenue, with US$394.3 billion in 2022 '
'revenue. As of March 2023, Apple is the '
"world's biggest...",
'descriptionSource': 'Wikipedia',
'descriptionLink': 'https://en.wikipedia.org/wiki/Apple_Inc.',
'attributes': {'Customer service': '1 (800) 275-2273',
'CEO': 'Tim Cook (Aug 24, 2011–)',
'Headquarters': 'Cupertino, CA',
'Founded': 'April 1, 1976, Los Altos, CA',
'Founders': 'Steve Jobs, Steve Wozniak, '
'Ronald Wayne, and more',
'Products': 'iPhone, iPad, Apple TV, and '
'more'}},
'organic': [{'title': 'Apple',
'link': 'https://www.apple.com/',
'snippet': 'Discover the innovative world of Apple and shop '
'everything iPhone, iPad, Apple Watch, Mac, and Apple '
'TV, plus explore accessories, entertainment, ...',


'TV, plus explore accessories, entertainment, ...',
'sitelinks': [{'title': 'Support',
'link': 'https://support.apple.com/'},
{'title': 'iPhone',
'link': 'https://www.apple.com/iphone/'},
{'title': 'Site Map',
'link': 'https://www.apple.com/sitemap/'},
{'title': 'Business',
'link': 'https://www.apple.com/business/'},
{'title': 'Mac',
'link': 'https://www.apple.com/mac/'},
{'title': 'Watch',
'link': 'https://www.apple.com/watch/'}],
'position': 1},
{'title': 'Apple Inc. - Wikipedia',
'link': 'https://en.wikipedia.org/wiki/Apple_Inc.',
'snippet': 'Apple Inc. is an American multinational technology '
'company headquartered in Cupertino, California. '
"Apple is the world's largest technology company by "
'revenue, ...',
'attributes': {'Products': 'AirPods; Apple Watch; iPad; iPhone; '
'Mac; Full list',
'Founders': 'Steve Jobs; Steve Wozniak; Ronald '
'Wayne; Mike Markkula'},
'sitelinks': [{'title': 'History',
'link': 'https://en.wikipedia.org/wiki/History_of_Apple_Inc.'},
{'title': 'Timeline of Apple Inc. products',
'link': 'https://en.wikipedia.org/wiki/Timeline_of_Apple_Inc._products'},
{'title': 'Litigation involving Apple Inc.',
'link': 'https://en.wikipedia.org/wiki/Litigation_involving_Apple_Inc.'},
{'title': 'Apple Store',
'link': 'https://en.wikipedia.org/wiki/Apple_Store'}],
'imageUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcRvmB5fT1LjqpZx02UM7IJq0Buoqt0DZs_y0dqwxwSWyP4PIN9FaxuTea0&s',
'position': 2},
{'title': 'Apple Inc. | History, Products, Headquarters, & Facts '
'| Britannica',


'| Britannica',
'link': 'https://www.britannica.com/topic/Apple-Inc',
'snippet': 'Apple Inc., formerly Apple Computer, Inc., American '
'manufacturer of personal computers, smartphones, '
'tablet computers, computer peripherals, and computer '
'...',
'attributes': {'Related People': 'Steve Jobs Steve Wozniak Jony '
'Ive Tim Cook Angela Ahrendts',
'Date': '1976 - present'},
'imageUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcS3liELlhrMz3Wpsox29U8jJ3L8qETR0hBWHXbFnwjwQc34zwZvFELst2E&s',
'position': 3},
{'title': 'AAPL: Apple Inc Stock Price Quote - NASDAQ GS - '
'Bloomberg.com',
'link': 'https://www.bloomberg.com/quote/AAPL:US',
'snippet': 'AAPL:USNASDAQ GS. Apple Inc. COMPANY INFO ; Open. '
'170.09 ; Prev Close. 169.59 ; Volume. 48,425,696 ; '
'Market Cap. 2.667T ; Day Range. 167.54170.35.',
'position': 4},
{'title': 'Apple Inc. (AAPL) Company Profile & Facts - Yahoo '
'Finance',
'link': 'https://finance.yahoo.com/quote/AAPL/profile/',
'snippet': 'Apple Inc. designs, manufactures, and markets '
'smartphones, personal computers, tablets, wearables, '
'and accessories worldwide. The company offers '
'iPhone, a line ...',
'position': 5},
{'title': 'Apple Inc. (AAPL) Stock Price, News, Quote & History - '
'Yahoo Finance',
'link': 'https://finance.yahoo.com/quote/AAPL',
'snippet': 'Find the latest Apple Inc. (AAPL) stock quote, '
'history, news and other vital information to help '
'you with your stock trading and investing.',
'position': 6}],


'you with your stock trading and investing.',
'position': 6}],
'peopleAlsoAsk': [{'question': 'What does Apple Inc do?',
'snippet': 'Apple Inc. (Apple) designs, manufactures and '
'markets smartphones, personal\n'
'computers, tablets, wearables and accessories '
'and sells a range of related\n'
'services.',
'title': 'AAPL.O - | Stock Price & Latest News - Reuters',
'link': 'https://www.reuters.com/markets/companies/AAPL.O/'},
{'question': 'What is the full form of Apple Inc?',
'snippet': '(formerly Apple Computer Inc.) is an American '
'computer and consumer electronics\n'
'company famous for creating the iPhone, iPad '
'and Macintosh computers.',
'title': 'What is Apple? An products and history overview '
'- TechTarget',
'link': 'https://www.techtarget.com/whatis/definition/Apple'},
{'question': 'What is Apple Inc iPhone?',
'snippet': 'Apple Inc (Apple) designs, manufactures, and '
'markets smartphones, tablets,\n'
'personal computers, and wearable devices. The '
'company also offers software\n'
'applications and related services, '
'accessories, and third-party digital content.\n'
"Apple's product portfolio includes iPhone, "
'iPad, Mac, iPod, Apple Watch, and\n'
'Apple TV.',
'title': 'Apple Inc Company Profile - Apple Inc Overview - '
'GlobalData',
'link': 'https://www.globaldata.com/company-profile/apple-inc/'},
{'question': 'Who runs Apple Inc?',
'snippet': 'Timothy Donald Cook (born November 1, 1960) is '
'an American business executive\n'
'who has been the chief executive officer of '
'Apple Inc. since 2011. Cook\n'
"previously served as the company's chief "
'operating officer under its co-founder\n'
'Steve Jobs. He is the first CEO of any Fortune '
'500 company who is openly gay.',
'title': 'Tim Cook - Wikipedia',
'link': 'https://en.wikipedia.org/wiki/Tim_Cook'}],


'link': 'https://en.wikipedia.org/wiki/Tim_Cook'}],
'relatedSearches': [{'query': 'Who invented the iPhone'},
{'query': 'Apple iPhone'},
{'query': 'History of Apple company PDF'},
{'query': 'Apple company history'},
{'query': 'Apple company introduction'},
{'query': 'Apple India'},
{'query': 'What does Apple Inc own'},
{'query': 'Apple Inc After Steve'},
{'query': 'Apple Watch'},
{'query': 'Apple App Store'}]}
We can also query Google Images using this wrapper. For example:
{'searchParameters': {'q': 'Lion',
'gl': 'us',
'hl': 'en',
'num': 10,
'type': 'images'},
'images': [{'title': 'Lion - Wikipedia',
'imageUrl': 'https://upload.wikimedia.org/wikipedia/commons/thumb/7/73/Lion_waiting_in_Namibia.jpg/1200px-Lion_waiting_in_Namibia.jpg',
'imageWidth': 1200,
'imageHeight': 900,
'thumbnailUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcRye79ROKwjfb6017jr0iu8Bz2E1KKuHg-A4qINJaspyxkZrkw&s',
'thumbnailWidth': 259,
'thumbnailHeight': 194,
'source': 'Wikipedia',
'domain': 'en.wikipedia.org',
'link': 'https://en.wikipedia.org/wiki/Lion',
'position': 1},
{'title': 'Lion | Characteristics, Habitat, & Facts | Britannica',
'imageUrl': 'https://cdn.britannica.com/55/2155-050-604F5A4A/lion.jpg',
'imageWidth': 754,
'imageHeight': 752,
'thumbnailUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcS3fnDub1GSojI0hJ-ZGS8Tv-hkNNloXh98DOwXZoZ_nUs3GWSd&s',
'thumbnailWidth': 225,
'thumbnailHeight': 224,


'thumbnailWidth': 225,
'thumbnailHeight': 224,
'source': 'Encyclopedia Britannica',
'domain': 'www.britannica.com',
'link': 'https://www.britannica.com/animal/lion',
'position': 2},
{'title': 'African lion, facts and photos',
'imageUrl': 'https://i.natgeofe.com/n/487a0d69-8202-406f-a6a0-939ed3704693/african-lion.JPG',
'imageWidth': 3072,
'imageHeight': 2043,
'thumbnailUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcTPlTarrtDbyTiEm-VI_PML9VtOTVPuDXJ5ybDf_lN11H2mShk&s',
'thumbnailWidth': 275,
'thumbnailHeight': 183,
'source': 'National Geographic',
'domain': 'www.nationalgeographic.com',
'link': 'https://www.nationalgeographic.com/animals/mammals/facts/african-lion',
'position': 3},
{'title': 'Saint Louis Zoo | African Lion',
'imageUrl': 'https://optimise2.assets-servd.host/maniacal-finch/production/animals/african-lion-01-01.jpg?w=1200&auto=compress%2Cformat&fit=crop&dm=1658933674&s=4b63f926a0f524f2087a8e0613282bdb',
'imageWidth': 1200,
'imageHeight': 1200,
'thumbnailUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcTlewcJ5SwC7yKup6ByaOjTnAFDeoOiMxyJTQaph2W_I3dnks4&s',
'thumbnailWidth': 225,
'thumbnailHeight': 225,
'source': 'St. Louis Zoo',
'domain': 'stlzoo.org',
'link': 'https://stlzoo.org/animals/mammals/carnivores/lion',


'position': 4},
{'title': 'How to Draw a Realistic Lion like an Artist - Studio '
'Wildlife',
'imageUrl': 'https://studiowildlife.com/wp-content/uploads/2021/10/245528858_183911853822648_6669060845725210519_n.jpg',
'imageWidth': 1431,
'imageHeight': 2048,
'thumbnailUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcTmn5HayVj3wqoBDQacnUtzaDPZzYHSLKUlIEcni6VB8w0mVeA&s',
'thumbnailWidth': 188,
'thumbnailHeight': 269,
'source': 'Studio Wildlife',
'domain': 'studiowildlife.com',
'link': 'https://studiowildlife.com/how-to-draw-a-realistic-lion-like-an-artist/',
'position': 5},
{'title': 'Lion | Characteristics, Habitat, & Facts | Britannica',
'imageUrl': 'https://cdn.britannica.com/29/150929-050-547070A1/lion-Kenya-Masai-Mara-National-Reserve.jpg',
'imageWidth': 1600,
'imageHeight': 1085,
'thumbnailUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcSCqaKY_THr0IBZN8c-2VApnnbuvKmnsWjfrwKoWHFR9w3eN5o&s',
'thumbnailWidth': 273,
'thumbnailHeight': 185,
'source': 'Encyclopedia Britannica',
'domain': 'www.britannica.com',
'link': 'https://www.britannica.com/animal/lion',
'position': 6},
{'title': "Where do lions live? Facts about lions' habitats and "
'other cool facts',


'other cool facts',
'imageUrl': 'https://www.gannett-cdn.com/-mm-/b2b05a4ab25f4fca0316459e1c7404c537a89702/c=0-0-1365-768/local/-/media/2022/03/16/USATODAY/usatsports/imageForEntry5-ODq.jpg?width=1365&height=768&fit=crop&format=pjpg&auto=webp',
'imageWidth': 1365,
'imageHeight': 768,
'thumbnailUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcTc_4vCHscgvFvYy3PSrtIOE81kNLAfhDK8F3mfOuotL0kUkbs&s',
'thumbnailWidth': 299,
'thumbnailHeight': 168,
'source': 'USA Today',
'domain': 'www.usatoday.com',
'link': 'https://www.usatoday.com/story/news/2023/01/08/where-do-lions-live-habitat/10927718002/',
'position': 7},
{'title': 'Lion',
'imageUrl': 'https://i.natgeofe.com/k/1d33938b-3d02-4773-91e3-70b113c3b8c7/lion-male-roar_square.jpg',
'imageWidth': 3072,
'imageHeight': 3072,
'thumbnailUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQqLfnBrBLcTiyTZynHH3FGbBtX2bd1ScwpcuOLnksTyS9-4GM&s',
'thumbnailWidth': 225,
'thumbnailHeight': 225,
'source': 'National Geographic Kids',
'domain': 'kids.nationalgeographic.com',
'link': 'https://kids.nationalgeographic.com/animals/mammals/facts/lion',
'position': 8},
{'title': "Lion | Smithsonian's National Zoo",


'position': 8},
{'title': "Lion | Smithsonian's National Zoo",
'imageUrl': 'https://nationalzoo.si.edu/sites/default/files/styles/1400_scale/public/animals/exhibit/africanlion-005.jpg?itok=6wA745g_',
'imageWidth': 1400,
'imageHeight': 845,
'thumbnailUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcSgB3z_D4dMEOWJ7lajJk4XaQSL4DdUvIRj4UXZ0YoE5fGuWuo&s',
'thumbnailWidth': 289,
'thumbnailHeight': 174,
'source': "Smithsonian's National Zoo",
'domain': 'nationalzoo.si.edu',
'link': 'https://nationalzoo.si.edu/animals/lion',
'position': 9},
{'title': "Zoo's New Male Lion Explores Habitat for the First Time "
'- Virginia Zoo',
'imageUrl': 'https://virginiazoo.org/wp-content/uploads/2022/04/ZOO_0056-scaled.jpg',
'imageWidth': 2560,
'imageHeight': 2141,
'thumbnailUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcTDCG7XvXRCwpe_-Vy5mpvrQpVl5q2qwgnDklQhrJpQzObQGz4&s',
'thumbnailWidth': 246,
'thumbnailHeight': 205,
'source': 'Virginia Zoo',
'domain': 'virginiazoo.org',
'link': 'https://virginiazoo.org/zoos-new-male-lion-explores-habitat-for-thefirst-time/',
'position': 10}]}
We can also query Google News using this wrapper. For example:
{'searchParameters': {'q': 'Tesla Inc.',
'gl': 'us',
'hl': 'en',
'num': 10,
'type': 'news'},
'news': [{'title': 'ISS recommends Tesla investors vote against re-election '
'of Robyn Denholm',


'of Robyn Denholm',
'link': 'https://www.reuters.com/business/autos-transportation/iss-recommends-tesla-investors-vote-against-re-election-robyn-denholm-2023-05-04/',
'snippet': 'Proxy advisory firm ISS on Wednesday recommended Tesla '
'investors vote against re-election of board chair Robyn '
'Denholm, citing "concerns on...',
'date': '5 mins ago',
'source': 'Reuters',
'imageUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcROdETe_GUyp1e8RHNhaRM8Z_vfxCvdfinZwzL1bT1ZGSYaGTeOojIdBoLevA&s',
'position': 1},
{'title': 'Global companies by market cap: Tesla fell most in April',
'link': 'https://www.reuters.com/markets/global-companies-by-market-cap-tesla-fell-most-april-2023-05-02/',
'snippet': 'Tesla Inc was the biggest loser among top companies by '
'market capitalisation in April, hit by disappointing '
'quarterly earnings after it...',
'date': '1 day ago',
'source': 'Reuters',
'imageUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQ4u4CP8aOdGyRFH6o4PkXi-_eZDeY96vLSag5gDjhKMYf98YBER2cZPbkStQ&s',
'position': 2},
{'title': 'Tesla Wanted an EV Price War. Ford Showed Up.',
'link': 'https://www.bloomberg.com/opinion/articles/2023-05-03/tesla-wanted-an-ev-price-war-ford-showed-up',
'snippet': 'The legacy automaker is paring back the cost of its '
'Mustang Mach-E model after Tesla discounted its '
'competing EVs, portending tighter...',
'date': '6 hours ago',
'source': 'Bloomberg.com',


'date': '6 hours ago',
'source': 'Bloomberg.com',
'imageUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcS_3Eo4VI0H-nTeIbYc5DaQn5ep7YrWnmhx6pv8XddFgNF5zRC9gEpHfDq8yQ&s',
'position': 3},
{'title': 'Joby Aviation to get investment from Tesla shareholder '
'Baillie Gifford',
'link': 'https://finance.yahoo.com/news/joby-aviation-investment-tesla-shareholder-204450712.html',
'snippet': 'This comes days after Joby clinched a $55 million '
'contract extension to deliver up to nine air taxis to '
'the U.S. Air Force,...',
'date': '4 hours ago',
'source': 'Yahoo Finance',
'imageUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQO0uVn297LI-xryrPNqJ-apUOulj4ohM-xkN4OfmvMOYh1CPdUEBbYx6hviw&s',
'position': 4},
{'title': 'Tesla resumes U.S. orders for a Model 3 version at lower '
'price, range',
'link': 'https://finance.yahoo.com/news/tesla-resumes-us-orders-model-045736115.html',
'snippet': '(Reuters) -Tesla Inc has resumed taking orders for its '
'Model 3 long-range vehicle in the United States, the '
"company's website showed late on...",
'date': '19 hours ago',
'source': 'Yahoo Finance',
'imageUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcTIZetJ62sQefPfbQ9KKDt6iH7Mc0ylT5t_hpgeeuUkHhJuAx2FOJ4ZTRVDFg&s',
'position': 5},
{'title': 'The Tesla Model 3 Long Range AWD Is Now Available in the '


{'title': 'The Tesla Model 3 Long Range AWD Is Now Available in the '
'U.S. With 325 Miles of Range',
'link': 'https://www.notateslaapp.com/news/1393/tesla-reopens-orders-for-model-3-long-range-after-months-of-unavailability',
'snippet': 'Tesla has reopened orders for the Model 3 Long Range '
'RWD, which has been unavailable for months due to high '
'demand.',
'date': '7 hours ago',
'source': 'Not a Tesla App',
'imageUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcSecrgxZpRj18xIJY-nDHljyP-A4ejEkswa9eq77qhMNrScnVIqe34uql5U4w&s',
'position': 6},
{'title': 'Tesla Cybertruck alpha prototype spotted at the Fremont '
'factory in new pics and videos',
'link': 'https://www.teslaoracle.com/2023/05/03/tesla-cybertruck-alpha-prototype-interior-and-exterior-spotted-at-the-fremont-factory-in-new-pics-and-videos/',
'snippet': 'A Tesla Cybertruck alpha prototype goes to Fremont, '
'California for another round of testing before going to '
'production later this year (pics...',
'date': '14 hours ago',
'source': 'Tesla Oracle',
'imageUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcRO7M5ZLQE-Zo4-_5dv9hNAQZ3wSqfvYCuKqzxHG-M6CgLpwPMMG_ssebdcMg&s',
'position': 7},
{'title': 'Tesla putting facility in new part of country - Austin '
'Business Journal',
'link': 'https://www.bizjournals.com/austin/news/2023/05/02/tesla-leases-building-seattle-area.html',
'snippet': 'Check out what Puget Sound Business Journal has to '
"report about the Austin-based company's real estate "
'footprint in the Pacific Northwest.',


'footprint in the Pacific Northwest.',
'date': '22 hours ago',
'source': 'The Business Journals',
'imageUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcR9kIEHWz1FcHKDUtGQBS0AjmkqtyuBkQvD8kyIY3kpaPrgYaN7I_H2zoOJsA&s',
'position': 8},
{'title': 'Tesla (TSLA) Resumes Orders for Model 3 Long Range After '
'Backlog',
'link': 'https://www.bloomberg.com/news/articles/2023-05-03/tesla-resumes-orders-for-popular-model-3-long-range-at-47-240',
'snippet': 'Tesla Inc. has resumed taking orders for its Model 3 '
'Long Range edition with a starting price of $47240, '
'according to its website.',
'date': '5 hours ago',
'source': 'Bloomberg.com',
'imageUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcTWWIC4VpMTfRvSyqiomODOoLg0xhoBf-Tc1qweKnSuaiTk-Y1wMJZM3jct0w&s',
'position': 9}]}
If you want to only receive news articles published in the last hour, you can do the following:
{'searchParameters': {'q': 'Tesla Inc.',
'gl': 'us',
'hl': 'en',
'num': 10,
'type': 'news',
'tbs': 'qdr:h'},
'news': [{'title': 'Oklahoma Gov. Stitt sees growing foreign interest in '
'investments in ...',
'link': 'https://www.reuters.com/world/us/oklahoma-gov-stitt-sees-growing-foreign-interest-investments-state-2023-05-04/',
'snippet': 'T)), a battery supplier to electric vehicle maker Tesla '
'Inc (TSLA.O), said on Sunday it is considering building '
'a battery plant in Oklahoma, its third in...',
'date': '53 mins ago',


'a battery plant in Oklahoma, its third in...',
'date': '53 mins ago',
'source': 'Reuters',
'imageUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcSSTcsXeenqmEKdiekvUgAmqIPR4nlAmgjTkBqLpza-lLfjX1CwB84MoNVj0Q&s',
'position': 1},
{'title': 'Ryder lanza solución llave en mano para vehículos '
'eléctricos en EU',
'link': 'https://www.tyt.com.mx/nota/ryder-lanza-solucion-llave-en-mano-para-vehiculos-electricos-en-eu',
'snippet': 'Ryder System Inc. presentó RyderElectric+ TM como su '
'nueva solución llave en mano ... Ryder también tiene '
'reservados los semirremolques Tesla y continúa...',
'date': '56 mins ago',
'source': 'Revista Transportes y Turismo',
'imageUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQJhXTQQtjSUZf9YPM235WQhFU5_d7lEA76zB8DGwZfixcgf1_dhPJyKA1Nbw&s',
'position': 2},
{'title': '"I think people can get by with $999 million," Bernie '
'Sanders tells American Billionaires.',
'link': 'https://thebharatexpressnews.com/i-think-people-can-get-by-with-999-million-bernie-sanders-tells-american-billionaires-heres-how-the-ultra-rich-can-pay-less-income-tax-than-you-legally/',
'snippet': 'The report noted that in 2007 and 2011, Amazon.com Inc. '
'founder Jeff Bezos “did not pay a dime in federal ... '
'If you want to bet on Musk, check out Tesla.',
'date': '11 mins ago',
'source': 'THE BHARAT EXPRESS NEWS',


'source': 'THE BHARAT EXPRESS NEWS',
'imageUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcR_X9qqSwVFBBdos2CK5ky5IWIE3aJPCQeRYR9O1Jz4t-MjaEYBuwK7AU3AJQ&s',
'position': 3}]}
qdr:h (past hour) qdr:d (past day) qdr:w (past week) qdr:m (past month) qdr:y (past year)
You can specify intermediate time periods by adding a number: qdr:h12 (past 12 hours) qdr:d3 (past 3 days) qdr:w2 (past 2 weeks) qdr:m6 (past 6 months) qdr:m2 (past 2 years)
For all supported filters simply go to Google Search, search for something, click on "Tools", add your date filter and check the URL for "tbs=".
We can also query Google Places using this wrapper. For example:
{'searchParameters': {'q': 'Italian restaurants in Upper East Side',
'gl': 'us',
'hl': 'en',
'num': 10,
'type': 'places'},
'places': [{'position': 1,
'title': "L'Osteria",
'address': '1219 Lexington Ave',
'latitude': 40.777154599999996,
'longitude': -73.9571363,
'thumbnailUrl': 'https://lh5.googleusercontent.com/p/AF1QipNjU7BWEq_aYQANBCbX52Kb0lDpd_lFIx5onw40=w92-h92-n-k-no',
'rating': 4.7,
'ratingCount': 91,
'category': 'Italian'},
{'position': 2,
'title': "Tony's Di Napoli",
'address': '1081 3rd Ave',
'latitude': 40.7643567,
'longitude': -73.9642373,


'longitude': -73.9642373,
'thumbnailUrl': 'https://lh5.googleusercontent.com/p/AF1QipNbNv6jZkJ9nyVi60__8c1DQbe_eEbugRAhIYye=w92-h92-n-k-no',
'rating': 4.5,
'ratingCount': 2265,
'category': 'Italian'},
{'position': 3,
'title': 'Caravaggio',
'address': '23 E 74th St',
'latitude': 40.773412799999996,
'longitude': -73.96473379999999,
'thumbnailUrl': 'https://lh5.googleusercontent.com/p/AF1QipPDGchokDvppoLfmVEo6X_bWd3Fz0HyxIHTEe9V=w92-h92-n-k-no',
'rating': 4.5,
'ratingCount': 276,
'category': 'Italian'},
{'position': 4,
'title': 'Luna Rossa',
'address': '347 E 85th St',
'latitude': 40.776593999999996,
'longitude': -73.950351,
'thumbnailUrl': 'https://lh5.googleusercontent.com/p/AF1QipNPCpCPuqPAb1Mv6_fOP7cjb8Wu1rbqbk2sMBlh=w92-h92-n-k-no',
'rating': 4.5,
'ratingCount': 140,
'category': 'Italian'},
{'position': 5,
'title': "Paola's",
'address': '1361 Lexington Ave',
'latitude': 40.7822019,
'longitude': -73.9534096,
'thumbnailUrl': 'https://lh5.googleusercontent.com/p/AF1QipPJr2Vcx-B6K-GNQa4koOTffggTePz8TKRTnWi3=w92-h92-n-k-no',
'rating': 4.5,
'ratingCount': 344,
'category': 'Italian'},
{'position': 6,
'title': 'Come Prima',


{'position': 6,
'title': 'Come Prima',
'address': '903 Madison Ave',
'latitude': 40.772124999999996,
'longitude': -73.965012,
'thumbnailUrl': 'https://lh5.googleusercontent.com/p/AF1QipNrX19G0NVdtDyMovCQ-M-m0c_gLmIxrWDQAAbz=w92-h92-n-k-no',
'rating': 4.5,
'ratingCount': 176,
'category': 'Italian'},
{'position': 7,
'title': 'Botte UES',
'address': '1606 1st Ave.',
'latitude': 40.7750785,
'longitude': -73.9504801,
'thumbnailUrl': 'https://lh5.googleusercontent.com/p/AF1QipPPN5GXxfH3NDacBc0Pt3uGAInd9OChS5isz9RF=w92-h92-n-k-no',
'rating': 4.4,
'ratingCount': 152,
'category': 'Italian'},
{'position': 8,
'title': 'Piccola Cucina Uptown',
'address': '106 E 60th St',
'latitude': 40.7632468,
'longitude': -73.9689825,
'thumbnailUrl': 'https://lh5.googleusercontent.com/p/AF1QipPifIgzOCD5SjgzzqBzGkdZCBp0MQsK5k7M7znn=w92-h92-n-k-no',
'rating': 4.6,
'ratingCount': 941,
'category': 'Italian'},
{'position': 9,
'title': 'Pinocchio Restaurant',
'address': '300 E 92nd St',
'latitude': 40.781453299999995,
'longitude': -73.9486788,
'thumbnailUrl': 'https://lh5.googleusercontent.com/p/AF1QipNtxlIyEEJHtDtFtTR9nB38S8A2VyMu-mVVz72A=w92-h92-n-k-no',


'rating': 4.5,
'ratingCount': 113,
'category': 'Italian'},
{'position': 10,
'title': 'Barbaresco',
'address': '843 Lexington Ave #1',
'latitude': 40.7654332,
'longitude': -73.9656873,
'thumbnailUrl': 'https://lh5.googleusercontent.com/p/AF1QipMb9FbPuXF_r9g5QseOHmReejxSHgSahPMPJ9-8=w92-h92-n-k-no',
'rating': 4.3,
'ratingCount': 122,
'locationHint': 'In The Touraine',
'category': 'Italian'}]}


Hugging Face Hub
Let's load the Hugging Face Embedding class.
from langchain.embeddings import HuggingFaceEmbeddings
embeddings = HuggingFaceEmbeddings()
text = "This is a test document."
query_result = embeddings.embed_query(text)
doc_result = embeddings.embed_documents([text])


GPT4All
GitHub:nomic-ai/gpt4all an ecosystem of open-source chatbots trained on a massive collections of clean assistant data including code, stories and dialogue.
This example goes over how to use LangChain to interact with GPT4All models.
%pip install gpt4all > /dev/null
Note: you may need to restart the kernel to use updated packages.
Import GPT4All
from langchain import PromptTemplate, LLMChain
from langchain.llms import GPT4All
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler
Set Up Question to pass to LLM
template = """Question: {question}

Answer: Let's think step by step."""

prompt = PromptTemplate(template=template, input_variables=["question"])
Specify Model
To run locally, download a compatible ggml-formatted model.
The gpt4all page has a useful Model Explorer section:
Select a model of interest
Download using the UI and move the .bin to the local_path (noted below)
For more info, visit https://github.com/nomic-ai/gpt4all.
local_path = (
"./models/ggml-gpt4all-l13b-snoozy.bin" # replace with your desired local file path
)
# Callbacks support token-wise streaming
callbacks = [StreamingStdOutCallbackHandler()]

# Verbose is required to pass to the callback manager
llm = GPT4All(model=local_path, callbacks=callbacks, verbose=True)

# If you want to use a custom model add the backend parameter
# Check https://docs.gpt4all.io/gpt4all_python.html for supported backends
llm = GPT4All(model=local_path, backend="gptj", callbacks=callbacks, verbose=True)
llm_chain = LLMChain(prompt=prompt, llm=llm)
question = "What NFL team won the Super Bowl in the year Justin Bieber was born?"

llm_chain.run(question)
Justin Bieber was born on March 1, 1994. In 1994, The Cowboys won Super Bowl XXVIII.


This example shows how one can track the following while calling OpenAI models via LangChain and Infino:
# Set your key here.
# os.environ["OPENAI_API_KEY"] = "YOUR_API_KEY"

# Create callback handler. This logs latency, errors, token usage, prompts as well as prompt responses to Infino.
handler = InfinoCallbackHandler(
model_id="test_openai", model_version="0.1", verbose=False
)

# Create LLM.
llm = OpenAI(temperature=0.1)

# Number of questions to ask the OpenAI model. We limit to a short number here to save $$ while running this demo.
num_questions = 10

questions = questions[0:num_questions]
for question in questions:
print(question)


# We send the question to OpenAI API, with Infino callback.
llm_result = llm.generate([question], callbacks=[handler])
print(llm_result)
In what country is Normandy located?
generations=[[Generation(text='\n\nNormandy is located in France.', generation_info={'finish_reason': 'stop', 'logprobs': None})]] llm_output={'token_usage': {'total_tokens': 16, 'completion_tokens': 9, 'prompt_tokens': 7}, 'model_name': 'text-davinci-003'} run=RunInfo(run_id=UUID('8de21639-acec-4bd1-a12d-8124de1e20da'))
When were the Normans in Normandy?
generations=[[Generation(text='\n\nThe Normans first settled in Normandy in the late 9th century.', generation_info={'finish_reason': 'stop', 'logprobs': None})]] llm_output={'token_usage': {'total_tokens': 24, 'completion_tokens': 16, 'prompt_tokens': 8}, 'model_name': 'text-davinci-003'} run=RunInfo(run_id=UUID('cf81fc86-250b-4e6e-9d92-2df3bebb019a'))
From which countries did the Norse originate?
generations=[[Generation(text='\n\nThe Norse originated from Scandinavia, which includes modern-day Norway, Sweden, and Denmark.', generation_info={'finish_reason': 'stop', 'logprobs': None})]] llm_output={'token_usage': {'total_tokens': 29, 'completion_tokens': 21, 'prompt_tokens': 8}, 'model_name': 'text-davinci-003'} run=RunInfo(run_id=UUID('50f42f5e-b4a4-411a-a049-f92cb573a74f'))
Who was the Norse leader?


Who was the Norse leader?
generations=[[Generation(text='\n\nThe most famous Norse leader was the legendary Viking king Ragnar Lodbrok. He is believed to have lived in the 9th century and is renowned for his exploits in England and France.', generation_info={'finish_reason': 'stop', 'logprobs': None})]] llm_output={'token_usage': {'total_tokens': 45, 'completion_tokens': 39, 'prompt_tokens': 6}, 'model_name': 'text-davinci-003'} run=RunInfo(run_id=UUID('e32f31cb-ddc9-4863-8e6e-cb7a281a0ada'))
What century did the Normans first gain their separate identity?
generations=[[Generation(text='\n\nThe Normans first gained their separate identity in the 11th century.', generation_info={'finish_reason': 'stop', 'logprobs': None})]] llm_output={'token_usage': {'total_tokens': 28, 'completion_tokens': 16, 'prompt_tokens': 12}, 'model_name': 'text-davinci-003'} run=RunInfo(run_id=UUID('da9d8f73-b3b3-4bc5-8495-da8b11462a51'))
Who gave their name to Normandy in the 1000's and 1100's
generations=[[Generation(text='\n\nThe Normans, a people from northern France, gave their name to Normandy in the 1000s and 1100s. The Normans were descended from Viking settlers who had come to the region in the late 800s.', generation_info={'finish_reason': 'stop', 'logprobs': None})]] llm_output={'token_usage': {'total_tokens': 58, 'completion_tokens': 45, 'prompt_tokens': 13}, 'model_name': 'text-davinci-003'} run=RunInfo(run_id=UUID('bb5829bf-b6a6-4429-adfa-414ac5be46e5'))
What is France a region of?


What is France a region of?
generations=[[Generation(text='\n\nFrance is a region of Europe.', generation_info={'finish_reason': 'stop', 'logprobs': None})]] llm_output={'token_usage': {'total_tokens': 16, 'completion_tokens': 9, 'prompt_tokens': 7}, 'model_name': 'text-davinci-003'} run=RunInfo(run_id=UUID('6943880b-b4e4-4c74-9ca1-8c03c10f7e9c'))
Who did King Charles III swear fealty to?
generations=[[Generation(text='\n\nKing Charles III swore fealty to Pope Innocent III.', generation_info={'finish_reason': 'stop', 'logprobs': None})]] llm_output={'token_usage': {'total_tokens': 23, 'completion_tokens': 13, 'prompt_tokens': 10}, 'model_name': 'text-davinci-003'} run=RunInfo(run_id=UUID('c91fd663-09e6-4d00-b746-4c7fd96f9ceb'))
When did the Frankish identity emerge?
generations=[[Generation(text='\n\nThe Frankish identity began to emerge in the late 5th century, when the Franks began to expand their power and influence in the region. The Franks were a Germanic tribe that had migrated to the area from the east and had established a kingdom in what is now modern-day France. The Franks were eventually able to establish a powerful kingdom that lasted until the 10th century.', generation_info={'finish_reason': 'stop', 'logprobs': None})]] llm_output={'token_usage': {'total_tokens': 86, 'completion_tokens': 78, 'prompt_tokens': 8}, 'model_name': 'text-davinci-003'} run=RunInfo(run_id=UUID('23f86775-e592-4cb8-baa3-46ebe74305b2'))
Who was the duke in the battle of Hastings?


Who was the duke in the battle of Hastings?
generations=[[Generation(text='\n\nThe Duke of Normandy, William the Conqueror, was the leader of the Norman forces at the Battle of Hastings in 1066.', generation_info={'finish_reason': 'stop', 'logprobs': None})]] llm_output={'token_usage': {'total_tokens': 39, 'completion_tokens': 28, 'prompt_tokens': 11}, 'model_name': 'text-davinci-003'} run=RunInfo(run_id=UUID('ad5b7984-8758-4d95-a5eb-ee56e0218f6b'))
We now use matplotlib to create graphs of latency, errors and tokens consumed.
# Helper function to create a graph using matplotlib.
def plot(data, title):
data = json.loads(data)


# Extract x and y values from the data
timestamps = [item["time"] for item in data]
dates = [dt.datetime.fromtimestamp(ts) for ts in timestamps]
y = [item["value"] for item in data]

plt.rcParams["figure.figsize"] = [6, 4]
plt.subplots_adjust(bottom=0.2)
plt.xticks(rotation=25)
ax = plt.gca()
xfmt = md.DateFormatter("%Y-%m-%d %H:%M:%S")
ax.xaxis.set_major_formatter(xfmt)

# Create the plot
plt.plot(dates, y)

# Set labels and title
plt.xlabel("Time")
plt.ylabel("Value")
plt.title(title)

plt.show()

response = client.search_ts("__name__", "latency", 0, int(time.time()))
plot(response.text, "Latency")

response = client.search_ts("__name__", "error", 0, int(time.time()))
plot(response.text, "Errors")

response = client.search_ts("__name__", "prompt_tokens", 0, int(time.time()))
plot(response.text, "Prompt Tokens")

response = client.search_ts("__name__", "completion_tokens", 0, int(time.time()))
plot(response.text, "Completion Tokens")

response = client.search_ts("__name__", "total_tokens", 0, int(time.time()))
plot(response.text, "Total Tokens")
# Search for a particular prompt text.
query = "normandy"
response = client.search_log(query, 0, int(time.time()))
print("Results for", query, ":", response.text)

print("===")

query = "king charles III"
response = client.search_log("king charles III", 0, int(time.time()))
print("Results for", query, ":", response.text)


Jina
Let's load the Jina Embedding class.
from langchain.embeddings import JinaEmbeddings
embeddings = JinaEmbeddings(
jina_auth_token=jina_auth_token, model_name="ViT-B-32::openai"
)
text = "This is a test document."
query_result = embeddings.embed_query(text)
doc_result = embeddings.embed_documents([text])
In the above example, ViT-B-32::openai, OpenAI's pretrained ViT-B-32 model is used. For a full list of models, see here.


This notebook shows how to use functionality related to the LanceDB vector database based on the Lance data format.
We want to use OpenAIEmbeddings so we have to get the OpenAI API Key.
import lancedb

db = lancedb.connect("/tmp/lancedb")
table = db.create_table(
"my_table",
data=[
{
"vector": embeddings.embed_query("Hello World"),
"text": "Hello World",
"id": "1",
}
],
mode="overwrite",
)

docsearch = LanceDB.from_documents(documents, embeddings, connection=table)

query = "What did the president say about Ketanji Brown Jackson"
docs = docsearch.similarity_search(query)
They were responding to a 9-1-1 call when a man shot and killed them with a stolen gun.

Officer Mora was 27 years old.

Officer Rivera was 22.

Both Dominican Americans who’d grown up on the same streets they later chose to patrol as police officers.

I spoke with their families and told them that we are forever in debt for their sacrifice, and we will carry on their mission to restore the trust and safety every community deserves.

I’ve worked on these issues a long time.

I know what works: Investing in crime preventionand community police officers who’ll walk the beat, who’ll know the neighborhood, and who can restore trust and safety.

So let’s not abandon our streets. Or choose between safety and equal justice.

Let’s come together to protect our communities, restore trust, and hold law enforcement accountable.

That’s why the Justice Department required body cameras, banned chokeholds, and restricted no-knock warrants for its officers.

That’s why the American Rescue Plan provided $350 Billion that cities, states, and counties can use to hire more police and invest in proven strategies like community violence interruption—trusted messengers breaking the cycle of violence and trauma and giving young people hope.

We should all agree: The answer is not to Defund the police. The answer is to FUND the police with the resources and training they need to protect our communities.

I ask Democrats and Republicans alike: Pass my budget and keep our neighborhoods safe.


I ask Democrats and Republicans alike: Pass my budget and keep our neighborhoods safe.

And I will keep doing everything in my power to crack down on gun trafficking and ghost guns you can buy online and make at home—they have no serial numbers and can’t be traced.

And I ask Congress to pass proven measures to reduce gun violence. Pass universal background checks. Why should anyone on a terrorist list be able to purchase a weapon?

Ban assault weapons and high-capacity magazines.

Repeal the liability shield that makes gun manufacturers the only industry in America that can’t be sued.

These laws don’t infringe on the Second Amendment. They save lives.

The most fundamental right in America is the right to vote – and to have it counted. And it’s under assault.

In state after state, new laws have been passed, not only to suppress the vote, but to subvert entire elections.

We cannot let this happen.

Tonight. I call on the Senate to: Pass the Freedom to Vote Act. Pass the John Lewis Voting Rights Act. And while you’re at it, pass the Disclose Act so Americans can know who is funding our elections.

Tonight, I’d like to honor someone who has dedicated his life to serve this country: Justice Stephen Breyer—an Army veteran, Constitutional scholar, and retiring Justice of the United States Supreme Court. Justice Breyer, thank you for your service.

One of the most serious constitutional responsibilities a President has is nominating someone to serve on the United States Supreme Court.

And I did that 4 days ago, when I nominated Circuit Court of Appeals Judge Ketanji Brown Jackson. One of our nation’s top legal minds, who will continue Justice Breyer’s legacy of excellence.

A former top litigator in private practice. A former federal public defender. And from a family of public school educators and police officers. A consensus builder. Since she’s been nominated, she’s received a broad range of support—from the Fraternal Order of Police to former judges appointed by Democrats and Republicans.

And if we are to advance liberty and justice, we need to secure the Border and fix the immigration system.

We can do both. At our border, we’ve installed new technology like cutting-edge scanners to better detect drug smuggling.


We’ve set up joint patrols with Mexico and Guatemala to catch more human traffickers.

We’re putting in place dedicated immigration judges so families fleeing persecution and violence can have their cases heard faster.


Llama.cpp
llama-cpp-python is a Python binding for llama.cpp.
It supports inference for many LLMs, which can be accessed on HuggingFace.
This notebook goes over how to run llama-cpp-python within LangChain.
Note: new versions of llama-cpp-python use GGUF model files (see here).
This is a breaking change.
To convert existing GGML models to GGUF you can run the following in llama.cpp:
python ./convert-llama-ggmlv3-to-gguf.py --eps 1e-5 --input models/openorca-platypus2-13b.ggmlv3.q4_0.bin --output models/openorca-platypus2-13b.gguf.q4_0.bin
Installation
There are different options on how to install the llama-cpp package:
CPU usage
CPU + GPU (using one of many BLAS backends)
Metal GPU (MacOS with Apple Silicon Chip)
CPU only installation
pip install llama-cpp-python
Installation with OpenBLAS / cuBLAS / CLBlast
lama.cpp supports multiple BLAS backends for faster processing. Use the FORCE_CMAKE=1 environment variable to force the use of cmake and install the pip package for the desired BLAS backend (source).
Example installation with cuBLAS backend:
CMAKE_ARGS="-DLLAMA_CUBLAS=on" FORCE_CMAKE=1 pip install llama-cpp-python
IMPORTANT: If you have already installed the CPU only version of the package, you need to reinstall it from scratch. Consider the following command:
CMAKE_ARGS="-DLLAMA_CUBLAS=on" FORCE_CMAKE=1 pip install --upgrade --force-reinstall llama-cpp-python --no-cache-dir
Installation with Metal
llama.cpp supports Apple silicon first-class citizen - optimized via ARM NEON, Accelerate and Metal frameworks. Use the FORCE_CMAKE=1 environment variable to force the use of cmake and install the pip package for the Metal support (source).
Example installation with Metal Support:
CMAKE_ARGS="-DLLAMA_METAL=on" FORCE_CMAKE=1 pip install llama-cpp-python
IMPORTANT: If you have already installed a cpu only version of the package, you need to reinstall it from scratch: consider the following command:


CMAKE_ARGS="-DLLAMA_METAL=on" FORCE_CMAKE=1 pip install --upgrade --force-reinstall llama-cpp-python --no-cache-dir
Installation with Windows
It is stable to install the llama-cpp-python library by compiling from the source. You can follow most of the instructions in the repository itself but there are some windows specific instructions which might be useful.
Requirements to install the llama-cpp-python,
git
python
cmake
Visual Studio Community (make sure you install this with the following settings)
Desktop development with C++
Python development
Linux embedded development with C++
Clone git repository recursively to get llama.cpp submodule as well
git clone --recursive -j8 https://github.com/abetlen/llama-cpp-python.git
Open up command Prompt (or anaconda prompt if you have it installed), set up environment variables to install. Follow this if you do not have a GPU, you must set both of the following variables.
set FORCE_CMAKE=1
set CMAKE_ARGS=-DLLAMA_CUBLAS=OFF
You can ignore the second environment variable if you have an NVIDIA GPU.
Compiling and installing
In the same command prompt (anaconda prompt) you set the variables, you can cd into llama-cpp-python directory and run the following commands.
python setup.py clean
python setup.py install
Usage
Make sure you are following all instructions to install all necessary model files.
You don't need an API_TOKEN as you will run the LLM locally.
It is worth understanding which models are suitable to be used on the desired machine.
from langchain.llms import LlamaCpp
from langchain import PromptTemplate, LLMChain
from langchain.callbacks.manager import CallbackManager
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler
Consider using a template that suits your model! Check the models page on HuggingFace etc. to get a correct prompting template.
template = """Question: {question}


Answer: Let's work this out in a step by step way to be sure we have the right answer."""

prompt = PromptTemplate(template=template, input_variables=["question"])
# Callbacks support token-wise streaming
callback_manager = CallbackManager([StreamingStdOutCallbackHandler()])
CPU
Example using a LLaMA 2 7B model
# Make sure the model path is correct for your system!
llm = LlamaCpp(
model_path="/Users/rlm/Desktop/Code/llama.cpp/models/openorca-platypus2-13b.gguf.q4_0.bin",
temperature=0.75,
max_tokens=2000,
top_p=1,
callback_manager=callback_manager,
verbose=True, # Verbose is required to pass to the callback manager
)
prompt = """
Question: A rap battle between Stephen Colbert and John Oliver
"""
llm(prompt)

Stephen Colbert:
Yo, John, I heard you've been talkin' smack about me on your show.
Let me tell you somethin', pal, I'm the king of late-night TV
My satire is sharp as a razor, it cuts deeper than a knife
While you're just a british bloke tryin' to be funny with your accent and your wit.
John Oliver:
Oh Stephen, don't be ridiculous, you may have the ratings but I got the real talk.
My show is the one that people actually watch and listen to, not just for the laughs but for the facts.
While you're busy talkin' trash, I'm out here bringing the truth to light.
Stephen Colbert:
Truth? Ha! You think your show is about truth? Please, it's all just a joke to you.
You're just a fancy-pants british guy tryin' to be funny with your news and your jokes.
While I'm the one who's really makin' a difference, with my sat


llama_print_timings: load time = 358.60 ms
llama_print_timings: sample time = 172.55 ms / 256 runs ( 0.67 ms per token, 1483.59 tokens per second)
llama_print_timings: prompt eval time = 613.36 ms / 16 tokens ( 38.33 ms per token, 26.09 tokens per second)
llama_print_timings: eval time = 10151.17 ms / 255 runs ( 39.81 ms per token, 25.12 tokens per second)
llama_print_timings: total time = 11332.41 ms

"\nStephen Colbert:\nYo, John, I heard you've been talkin' smack about me on your show.\nLet me tell you somethin', pal, I'm the king of late-night TV\nMy satire is sharp as a razor, it cuts deeper than a knife\nWhile you're just a british bloke tryin' to be funny with your accent and your wit.\nJohn Oliver:\nOh Stephen, don't be ridiculous, you may have the ratings but I got the real talk.\nMy show is the one that people actually watch and listen to, not just for the laughs but for the facts.\nWhile you're busy talkin' trash, I'm out here bringing the truth to light.\nStephen Colbert:\nTruth? Ha! You think your show is about truth? Please, it's all just a joke to you.\nYou're just a fancy-pants british guy tryin' to be funny with your news and your jokes.\nWhile I'm the one who's really makin' a difference, with my sat"
Example using a LLaMA v1 model
# Make sure the model path is correct for your system!
llm = LlamaCpp(
model_path="./ggml-model-q4_0.bin", callback_manager=callback_manager, verbose=True
)
llm_chain = LLMChain(prompt=prompt, llm=llm)
question = "What NFL team won the Super Bowl in the year Justin Bieber was born?"
llm_chain.run(question)


1. First, find out when Justin Bieber was born.
2. We know that Justin Bieber was born on March 1, 1994.
3. Next, we need to look up when the Super Bowl was played in that year.
4. The Super Bowl was played on January 28, 1995.
5. Finally, we can use this information to answer the question. The NFL team that won the Super Bowl in the year Justin Bieber was born is the San Francisco 49ers.

llama_print_timings: load time = 434.15 ms
llama_print_timings: sample time = 41.81 ms / 121 runs ( 0.35 ms per token)
llama_print_timings: prompt eval time = 2523.78 ms / 48 tokens ( 52.58 ms per token)
llama_print_timings: eval time = 23971.57 ms / 121 runs ( 198.11 ms per token)
llama_print_timings: total time = 28945.95 ms

'\n\n1. First, find out when Justin Bieber was born.\n2. We know that Justin Bieber was born on March 1, 1994.\n3. Next, we need to look up when the Super Bowl was played in that year.\n4. The Super Bowl was played on January 28, 1995.\n5. Finally, we can use this information to answer the question. The NFL team that won the Super Bowl in the year Justin Bieber was born is the San Francisco 49ers.'
GPU
If the installation with BLAS backend was correct, you will see a BLAS = 1 indicator in model properties.
Two of the most important parameters for use with GPU are:
n_gpu_layers - determines how many layers of the model are offloaded to your GPU.
n_batch - how many tokens are processed in parallel.
Setting these parameters correctly will dramatically improve the evaluation speed (see wrapper code for more details).
n_gpu_layers = 40 # Change this value based on your model and your GPU VRAM pool.
n_batch = 512 # Should be between 1 and n_ctx, consider the amount of VRAM in your GPU.


# Make sure the model path is correct for your system!
llm = LlamaCpp(
model_path="/Users/rlm/Desktop/Code/llama.cpp/models/openorca-platypus2-13b.gguf.q4_0.bin",
n_gpu_layers=n_gpu_layers,
n_batch=n_batch,
callback_manager=callback_manager,
verbose=True, # Verbose is required to pass to the callback manager
)
llm_chain = LLMChain(prompt=prompt, llm=llm)
question = "What NFL team won the Super Bowl in the year Justin Bieber was born?"
llm_chain.run(question)

1. Identify Justin Bieber's birth date: Justin Bieber was born on March 1, 1994.

2. Find the Super Bowl winner of that year: The NFL season of 1993 with the Super Bowl being played in January or of 1994.

3. Determine which team won the game: The Dallas Cowboys faced the Buffalo Bills in Super Bowl XXVII on January 31, 1993 (as the year is mis-labelled due to a error). The Dallas Cowboys won this matchup.

So, Justin Bieber was born when the Dallas Cowboys were the reigning NFL Super Bowl.

llama_print_timings: load time = 427.63 ms
llama_print_timings: sample time = 115.85 ms / 164 runs ( 0.71 ms per token, 1415.67 tokens per second)
llama_print_timings: prompt eval time = 427.53 ms / 45 tokens ( 9.50 ms per token, 105.26 tokens per second)
llama_print_timings: eval time = 4526.53 ms / 163 runs ( 27.77 ms per token, 36.01 tokens per second)
llama_print_timings: total time = 5293.77 ms