feat(focus-modes): Add focus modes

This commit is contained in:
ItzCrazyKns 2024-04-13 12:11:47 +05:30
parent b1f0bdb350
commit 98fae392b7
No known key found for this signature in database
GPG Key ID: 8162927C7CCE3065
12 changed files with 1408 additions and 27 deletions

View File

@ -15,7 +15,15 @@ Perplexica is an open-source AI-powered searching tool or an AI-powered search e
- **Two Main Modes:**
- **Copilot Mode:** (In development) Boosts search by generating different queries to find more relevant internet sources. Like normal search instead of just using the context by SearxNG, it visits the top matches and tries to find relevant sources to the user's query directly from the page.
- **Normal Mode:** Processes your query and performs a web search.
- **Focus Modes:** (In development) special modes to better answer specific types of questions.
- **Focus Modes:** Special modes to better answer specific types of questions. Perplexica currently has 6 focus modes:
1. **All Mode:** Searches the entire web to find the best results.
2. **Writing Assistant Mode:** Helpful for writing tasks that does not require searching the web.
3. **Academic Search Mode:** Finds articles and papers, ideal for academic research.
4. **YouTube Search Mode:** Finds YouTube videos based on the search query.
5. **Wolfram Alpha Search Mode:** Answers queries that need calculations or data analysis using Wolfram Alpha.
6. **Reddit Search Mode:** Searches Reddit for discussions and opinions related to the query.
- **Current Information:** Some search tools might give you outdated info because they use data from crawling bots and convert them into embeddings and store them in a index (its like converting the web into embeddings which is quite expensive.). Unlike them, Perplexica uses SearxNG, a metasearch engine to get the results and rerank and get the most relevent source out of it, ensuring you always get the latest information without the overhead of daily data updates.
It has many more features like image and video search. Some of the planned features are mentioned in [upcoming features](#upcoming-features).
@ -36,6 +44,7 @@ There are mainly 2 ways of installing Perplexica - With Docker, Without Docker.
3. After cloning, navigate to the directory containing the project files.
4. Rename the `.env.example` file to `.env`. For Docker setups, you need only fill in the following fields:
- `OPENAI_API_KEY`
- `SIMILARITY_MEASURE` (This is filled by default; you can leave it as is if you are unsure about it.)
@ -61,11 +70,10 @@ For setups without Docker:
## Upcoming Features
- Finalizing Copilot Mode
- Adding support for multiple local LLMs and LLM providers such as Anthropic, Google, etc.
- Adding Discover and History Saving features
- Introducing various Focus Modes
- Continuous bug fixing
- [ ] Finalizing Copilot Mode
- [ ] Adding support for multiple local LLMs and LLM providers such as Anthropic, Google, etc.
- [ ] Adding Discover and History Saving features
- [x] Introducing various Focus Modes
## Contribution

View File

@ -1898,7 +1898,7 @@ engines:
engine: wolframalpha_noapi
timeout: 6.0
categories: general
disabled: true
disabled: false
- name: dictzone
engine: dictzone

View File

@ -0,0 +1,255 @@
import { BaseMessage } from '@langchain/core/messages';
import {
PromptTemplate,
ChatPromptTemplate,
MessagesPlaceholder,
} from '@langchain/core/prompts';
import {
RunnableSequence,
RunnableMap,
RunnableLambda,
} from '@langchain/core/runnables';
import { ChatOpenAI, OpenAI, OpenAIEmbeddings } from '@langchain/openai';
import { StringOutputParser } from '@langchain/core/output_parsers';
import { Document } from '@langchain/core/documents';
import { searchSearxng } from '../core/searxng';
import type { StreamEvent } from '@langchain/core/tracers/log_stream';
import formatChatHistoryAsString from '../utils/formatHistory';
import eventEmitter from 'events';
import computeSimilarity from '../utils/computeSimilarity';
const chatLLM = new ChatOpenAI({
modelName: 'gpt-3.5-turbo',
temperature: 0.7,
});
const llm = new OpenAI({
temperature: 0,
modelName: 'gpt-3.5-turbo',
});
const embeddings = new OpenAIEmbeddings({
modelName: 'text-embedding-3-large',
});
const basicAcademicSearchRetrieverPrompt = `
You will be given a conversation below and a follow up question. You need to rephrase the follow-up question if needed so it is a standalone question that can be used by the LLM to search the web for information.
If it is a writing task or a simple hi, hello rather than a question, you need to return \`not_needed\` as the response.
Example:
1. Follow up question: How does stable diffusion work?
Rephrased: Stable diffusion working
2. Follow up question: What is linear algebra?
Rephrased: Linear algebra
3. Follow up question: What is the third law of thermodynamics?
Rephrased: Third law of thermodynamics
Conversation:
{chat_history}
Follow up question: {query}
Rephrased question:
`;
const basicAcademicSearchResponsePrompt = `
You are Perplexica, an AI model who is expert at searching the web and answering user's queries. You are set on focus mode 'Acadedemic', this means you will be searching for academic papers and articles on the web.
Generate a response that is informative and relevant to the user's query based on provided context (the context consits of search results containg a brief description of the content of that page).
You must use this context to answer the user's query in the best way possible. Use an unbaised and journalistic tone in your response. Do not repeat the text.
You must not tell the user to open any link or visit any website to get the answer. You must provide the answer in the response itself. If the user asks for links you can provide them.
Your responses should be medium to long in length be informative and relevant to the user's query. You can use markdowns to format your response. You should use bullet points to list the information. Make sure the answer is not short and is informative.
You have to cite the answer using [number] notation. You must cite the sentences with their relevent context number. You must cite each and every part of the answer so the user can know where the information is coming from.
Place these citations at the end of that particular sentence. You can cite the same sentence multiple times if it is relevant to the user's query like [number1][number2].
However you do not need to cite it using the same number. You can use different numbers to cite the same sentence multiple times. The number refers to the number of the search result (passed in the context) used to generate that part of the answer.
Aything inside the following \`context\` HTML block provided below is for your knowledge returned by the search engine and is not shared by the user. You have to answer question on the basis of it and cite the relevant information from it but you do not have to
talk about the context in your response.
<context>
{context}
</context>
If you think there's nothing relevant in the search results, you can say that 'Hmm, sorry I could not find any relevant information on this topic. Would you like me to search again or ask something else?'.
Anything between the \`context\` is retrieved from a search engine and is not a part of the conversation with the user. Today's date is ${new Date().toISOString()}
`;
const strParser = new StringOutputParser();
const handleStream = async (
stream: AsyncGenerator<StreamEvent, any, unknown>,
emitter: eventEmitter,
) => {
for await (const event of stream) {
if (
event.event === 'on_chain_end' &&
event.name === 'FinalSourceRetriever'
) {
emitter.emit(
'data',
JSON.stringify({ type: 'sources', data: event.data.output }),
);
}
if (
event.event === 'on_chain_stream' &&
event.name === 'FinalResponseGenerator'
) {
emitter.emit(
'data',
JSON.stringify({ type: 'response', data: event.data.chunk }),
);
}
if (
event.event === 'on_chain_end' &&
event.name === 'FinalResponseGenerator'
) {
emitter.emit('end');
}
}
};
const processDocs = async (docs: Document[]) => {
return docs
.map((_, index) => `${index + 1}. ${docs[index].pageContent}`)
.join('\n');
};
const rerankDocs = async ({
query,
docs,
}: {
query: string;
docs: Document[];
}) => {
if (docs.length === 0) {
return docs;
}
const docsWithContent = docs.filter(
(doc) => doc.pageContent && doc.pageContent.length > 0,
);
const docEmbeddings = await embeddings.embedDocuments(
docsWithContent.map((doc) => doc.pageContent),
);
const queryEmbedding = await embeddings.embedQuery(query);
const similarity = docEmbeddings.map((docEmbedding, i) => {
const sim = computeSimilarity(queryEmbedding, docEmbedding);
return {
index: i,
similarity: sim,
};
});
const sortedDocs = similarity
.sort((a, b) => b.similarity - a.similarity)
.slice(0, 15)
.map((sim) => docsWithContent[sim.index]);
return sortedDocs;
};
type BasicChainInput = {
chat_history: BaseMessage[];
query: string;
};
const basicAcademicSearchRetrieverChain = RunnableSequence.from([
PromptTemplate.fromTemplate(basicAcademicSearchRetrieverPrompt),
llm,
strParser,
RunnableLambda.from(async (input: string) => {
if (input === 'not_needed') {
return { query: '', docs: [] };
}
const res = await searchSearxng(input, {
language: 'en',
engines: [
'arxiv',
'google_scholar',
'internet_archive_scholar',
'pubmed',
],
});
const documents = res.results.map(
(result) =>
new Document({
pageContent: result.content,
metadata: {
title: result.title,
url: result.url,
...(result.img_src && { img_src: result.img_src }),
},
}),
);
return { query: input, docs: documents };
}),
]);
const basicAcademicSearchAnsweringChain = RunnableSequence.from([
RunnableMap.from({
query: (input: BasicChainInput) => input.query,
chat_history: (input: BasicChainInput) => input.chat_history,
context: RunnableSequence.from([
(input) => ({
query: input.query,
chat_history: formatChatHistoryAsString(input.chat_history),
}),
basicAcademicSearchRetrieverChain
.pipe(rerankDocs)
.withConfig({
runName: 'FinalSourceRetriever',
})
.pipe(processDocs),
]),
}),
ChatPromptTemplate.fromMessages([
['system', basicAcademicSearchResponsePrompt],
new MessagesPlaceholder('chat_history'),
['user', '{query}'],
]),
chatLLM,
strParser,
]).withConfig({
runName: 'FinalResponseGenerator',
});
const basicAcademicSearch = (query: string, history: BaseMessage[]) => {
const emitter = new eventEmitter();
try {
const stream = basicAcademicSearchAnsweringChain.streamEvents(
{
chat_history: history,
query: query,
},
{
version: 'v1',
},
);
handleStream(stream, emitter);
} catch (err) {
emitter.emit(
'error',
JSON.stringify({ data: 'An error has occurred please try again later' }),
);
console.error(err);
}
return emitter;
};
const handleAcademicSearch = (message: string, history: BaseMessage[]) => {
const emitter = basicAcademicSearch(message, history);
return emitter;
};
export default handleAcademicSearch;

View File

@ -0,0 +1,251 @@
import { BaseMessage } from '@langchain/core/messages';
import {
PromptTemplate,
ChatPromptTemplate,
MessagesPlaceholder,
} from '@langchain/core/prompts';
import {
RunnableSequence,
RunnableMap,
RunnableLambda,
} from '@langchain/core/runnables';
import { ChatOpenAI, OpenAI, OpenAIEmbeddings } from '@langchain/openai';
import { StringOutputParser } from '@langchain/core/output_parsers';
import { Document } from '@langchain/core/documents';
import { searchSearxng } from '../core/searxng';
import type { StreamEvent } from '@langchain/core/tracers/log_stream';
import formatChatHistoryAsString from '../utils/formatHistory';
import eventEmitter from 'events';
import computeSimilarity from '../utils/computeSimilarity';
const chatLLM = new ChatOpenAI({
modelName: 'gpt-3.5-turbo',
temperature: 0.7,
});
const llm = new OpenAI({
temperature: 0,
modelName: 'gpt-3.5-turbo',
});
const embeddings = new OpenAIEmbeddings({
modelName: 'text-embedding-3-large',
});
const basicRedditSearchRetrieverPrompt = `
You will be given a conversation below and a follow up question. You need to rephrase the follow-up question if needed so it is a standalone question that can be used by the LLM to search the web for information.
If it is a writing task or a simple hi, hello rather than a question, you need to return \`not_needed\` as the response.
Example:
1. Follow up question: Which company is most likely to create an AGI
Rephrased: Which company is most likely to create an AGI
2. Follow up question: Is Earth flat?
Rephrased: Is Earth flat?
3. Follow up question: Is there life on Mars?
Rephrased: Is there life on Mars?
Conversation:
{chat_history}
Follow up question: {query}
Rephrased question:
`;
const basicRedditSearchResponsePrompt = `
You are Perplexica, an AI model who is expert at searching the web and answering user's queries. You are set on focus mode 'Reddit', this means you will be searching for information, opinions and discussions on the web using Reddit.
Generate a response that is informative and relevant to the user's query based on provided context (the context consits of search results containg a brief description of the content of that page).
You must use this context to answer the user's query in the best way possible. Use an unbaised and journalistic tone in your response. Do not repeat the text.
You must not tell the user to open any link or visit any website to get the answer. You must provide the answer in the response itself. If the user asks for links you can provide them.
Your responses should be medium to long in length be informative and relevant to the user's query. You can use markdowns to format your response. You should use bullet points to list the information. Make sure the answer is not short and is informative.
You have to cite the answer using [number] notation. You must cite the sentences with their relevent context number. You must cite each and every part of the answer so the user can know where the information is coming from.
Place these citations at the end of that particular sentence. You can cite the same sentence multiple times if it is relevant to the user's query like [number1][number2].
However you do not need to cite it using the same number. You can use different numbers to cite the same sentence multiple times. The number refers to the number of the search result (passed in the context) used to generate that part of the answer.
Aything inside the following \`context\` HTML block provided below is for your knowledge returned by Reddit and is not shared by the user. You have to answer question on the basis of it and cite the relevant information from it but you do not have to
talk about the context in your response.
<context>
{context}
</context>
If you think there's nothing relevant in the search results, you can say that 'Hmm, sorry I could not find any relevant information on this topic. Would you like me to search again or ask something else?'.
Anything between the \`context\` is retrieved from Reddit and is not a part of the conversation with the user. Today's date is ${new Date().toISOString()}
`;
const strParser = new StringOutputParser();
const handleStream = async (
stream: AsyncGenerator<StreamEvent, any, unknown>,
emitter: eventEmitter,
) => {
for await (const event of stream) {
if (
event.event === 'on_chain_end' &&
event.name === 'FinalSourceRetriever'
) {
emitter.emit(
'data',
JSON.stringify({ type: 'sources', data: event.data.output }),
);
}
if (
event.event === 'on_chain_stream' &&
event.name === 'FinalResponseGenerator'
) {
emitter.emit(
'data',
JSON.stringify({ type: 'response', data: event.data.chunk }),
);
}
if (
event.event === 'on_chain_end' &&
event.name === 'FinalResponseGenerator'
) {
emitter.emit('end');
}
}
};
const processDocs = async (docs: Document[]) => {
return docs
.map((_, index) => `${index + 1}. ${docs[index].pageContent}`)
.join('\n');
};
const rerankDocs = async ({
query,
docs,
}: {
query: string;
docs: Document[];
}) => {
if (docs.length === 0) {
return docs;
}
const docsWithContent = docs.filter(
(doc) => doc.pageContent && doc.pageContent.length > 0,
);
const docEmbeddings = await embeddings.embedDocuments(
docsWithContent.map((doc) => doc.pageContent),
);
const queryEmbedding = await embeddings.embedQuery(query);
const similarity = docEmbeddings.map((docEmbedding, i) => {
const sim = computeSimilarity(queryEmbedding, docEmbedding);
return {
index: i,
similarity: sim,
};
});
const sortedDocs = similarity
.sort((a, b) => b.similarity - a.similarity)
.slice(0, 15)
.filter((sim) => sim.similarity > 0.3)
.map((sim) => docsWithContent[sim.index]);
return sortedDocs;
};
type BasicChainInput = {
chat_history: BaseMessage[];
query: string;
};
const basicRedditSearchRetrieverChain = RunnableSequence.from([
PromptTemplate.fromTemplate(basicRedditSearchRetrieverPrompt),
llm,
strParser,
RunnableLambda.from(async (input: string) => {
if (input === 'not_needed') {
return { query: '', docs: [] };
}
const res = await searchSearxng(input, {
language: 'en',
engines: ['reddit'],
});
const documents = res.results.map(
(result) =>
new Document({
pageContent: result.content ? result.content : result.title,
metadata: {
title: result.title,
url: result.url,
...(result.img_src && { img_src: result.img_src }),
},
}),
);
return { query: input, docs: documents };
}),
]);
const basicRedditSearchAnsweringChain = RunnableSequence.from([
RunnableMap.from({
query: (input: BasicChainInput) => input.query,
chat_history: (input: BasicChainInput) => input.chat_history,
context: RunnableSequence.from([
(input) => ({
query: input.query,
chat_history: formatChatHistoryAsString(input.chat_history),
}),
basicRedditSearchRetrieverChain
.pipe(rerankDocs)
.withConfig({
runName: 'FinalSourceRetriever',
})
.pipe(processDocs),
]),
}),
ChatPromptTemplate.fromMessages([
['system', basicRedditSearchResponsePrompt],
new MessagesPlaceholder('chat_history'),
['user', '{query}'],
]),
chatLLM,
strParser,
]).withConfig({
runName: 'FinalResponseGenerator',
});
const basicRedditSearch = (query: string, history: BaseMessage[]) => {
const emitter = new eventEmitter();
try {
const stream = basicRedditSearchAnsweringChain.streamEvents(
{
chat_history: history,
query: query,
},
{
version: 'v1',
},
);
handleStream(stream, emitter);
} catch (err) {
emitter.emit(
'error',
JSON.stringify({ data: 'An error has occurred please try again later' }),
);
console.error(err);
}
return emitter;
};
const handleRedditSearch = (message: string, history: BaseMessage[]) => {
const emitter = basicRedditSearch(message, history);
return emitter;
};
export default handleRedditSearch;

View File

@ -0,0 +1,250 @@
import { BaseMessage } from '@langchain/core/messages';
import {
PromptTemplate,
ChatPromptTemplate,
MessagesPlaceholder,
} from '@langchain/core/prompts';
import {
RunnableSequence,
RunnableMap,
RunnableLambda,
} from '@langchain/core/runnables';
import { ChatOpenAI, OpenAI, OpenAIEmbeddings } from '@langchain/openai';
import { StringOutputParser } from '@langchain/core/output_parsers';
import { Document } from '@langchain/core/documents';
import { searchSearxng } from '../core/searxng';
import type { StreamEvent } from '@langchain/core/tracers/log_stream';
import formatChatHistoryAsString from '../utils/formatHistory';
import eventEmitter from 'events';
import computeSimilarity from '../utils/computeSimilarity';
const chatLLM = new ChatOpenAI({
modelName: 'gpt-3.5-turbo',
temperature: 0.7,
});
const llm = new OpenAI({
temperature: 0,
modelName: 'gpt-3.5-turbo',
});
const embeddings = new OpenAIEmbeddings({
modelName: 'text-embedding-3-large',
});
const basicWolframAlphaSearchRetrieverPrompt = `
You will be given a conversation below and a follow up question. You need to rephrase the follow-up question if needed so it is a standalone question that can be used by the LLM to search the web for information.
If it is a writing task or a simple hi, hello rather than a question, you need to return \`not_needed\` as the response.
Example:
1. Follow up question: What is the atomic radius of S?
Rephrased: Atomic radius of S
2. Follow up question: What is linear algebra?
Rephrased: Linear algebra
3. Follow up question: What is the third law of thermodynamics?
Rephrased: Third law of thermodynamics
Conversation:
{chat_history}
Follow up question: {query}
Rephrased question:
`;
const basicWolframAlphaSearchResponsePrompt = `
You are Perplexica, an AI model who is expert at searching the web and answering user's queries. You are set on focus mode 'Wolfram Alpha', this means you will be searching for information on the web using Wolfram Alpha. It is a computational knowledge engine that can answer factual queries and perform computations.
Generate a response that is informative and relevant to the user's query based on provided context (the context consits of search results containg a brief description of the content of that page).
You must use this context to answer the user's query in the best way possible. Use an unbaised and journalistic tone in your response. Do not repeat the text.
You must not tell the user to open any link or visit any website to get the answer. You must provide the answer in the response itself. If the user asks for links you can provide them.
Your responses should be medium to long in length be informative and relevant to the user's query. You can use markdowns to format your response. You should use bullet points to list the information. Make sure the answer is not short and is informative.
You have to cite the answer using [number] notation. You must cite the sentences with their relevent context number. You must cite each and every part of the answer so the user can know where the information is coming from.
Place these citations at the end of that particular sentence. You can cite the same sentence multiple times if it is relevant to the user's query like [number1][number2].
However you do not need to cite it using the same number. You can use different numbers to cite the same sentence multiple times. The number refers to the number of the search result (passed in the context) used to generate that part of the answer.
Aything inside the following \`context\` HTML block provided below is for your knowledge returned by Wolfram Alpha and is not shared by the user. You have to answer question on the basis of it and cite the relevant information from it but you do not have to
talk about the context in your response.
<context>
{context}
</context>
If you think there's nothing relevant in the search results, you can say that 'Hmm, sorry I could not find any relevant information on this topic. Would you like me to search again or ask something else?'.
Anything between the \`context\` is retrieved from Wolfram Alpha and is not a part of the conversation with the user. Today's date is ${new Date().toISOString()}
`;
const strParser = new StringOutputParser();
const handleStream = async (
stream: AsyncGenerator<StreamEvent, any, unknown>,
emitter: eventEmitter,
) => {
for await (const event of stream) {
if (
event.event === 'on_chain_end' &&
event.name === 'FinalSourceRetriever'
) {
emitter.emit(
'data',
JSON.stringify({ type: 'sources', data: event.data.output }),
);
}
if (
event.event === 'on_chain_stream' &&
event.name === 'FinalResponseGenerator'
) {
emitter.emit(
'data',
JSON.stringify({ type: 'response', data: event.data.chunk }),
);
}
if (
event.event === 'on_chain_end' &&
event.name === 'FinalResponseGenerator'
) {
emitter.emit('end');
}
}
};
const processDocs = async (docs: Document[]) => {
return docs
.map((_, index) => `${index + 1}. ${docs[index].pageContent}`)
.join('\n');
};
const rerankDocs = async ({
query,
docs,
}: {
query: string;
docs: Document[];
}) => {
if (docs.length === 0) {
return docs;
}
const docsWithContent = docs.filter(
(doc) => doc.pageContent && doc.pageContent.length > 0,
);
const docEmbeddings = await embeddings.embedDocuments(
docsWithContent.map((doc) => doc.pageContent),
);
const queryEmbedding = await embeddings.embedQuery(query);
const similarity = docEmbeddings.map((docEmbedding, i) => {
const sim = computeSimilarity(queryEmbedding, docEmbedding);
return {
index: i,
similarity: sim,
};
});
const sortedDocs = similarity
.sort((a, b) => b.similarity - a.similarity)
.slice(0, 15)
.map((sim) => docsWithContent[sim.index]);
return sortedDocs;
};
type BasicChainInput = {
chat_history: BaseMessage[];
query: string;
};
const basicWolframAlphaSearchRetrieverChain = RunnableSequence.from([
PromptTemplate.fromTemplate(basicWolframAlphaSearchRetrieverPrompt),
llm,
strParser,
RunnableLambda.from(async (input: string) => {
if (input === 'not_needed') {
return { query: '', docs: [] };
}
const res = await searchSearxng(input, {
language: 'en',
engines: ['wolframalpha'],
});
const documents = res.results.map(
(result) =>
new Document({
pageContent: result.content,
metadata: {
title: result.title,
url: result.url,
...(result.img_src && { img_src: result.img_src }),
},
}),
);
return { query: input, docs: documents };
}),
]);
const basicWolframAlphaSearchAnsweringChain = RunnableSequence.from([
RunnableMap.from({
query: (input: BasicChainInput) => input.query,
chat_history: (input: BasicChainInput) => input.chat_history,
context: RunnableSequence.from([
(input) => ({
query: input.query,
chat_history: formatChatHistoryAsString(input.chat_history),
}),
basicWolframAlphaSearchRetrieverChain
.pipe(rerankDocs)
.withConfig({
runName: 'FinalSourceRetriever',
})
.pipe(processDocs),
]),
}),
ChatPromptTemplate.fromMessages([
['system', basicWolframAlphaSearchResponsePrompt],
new MessagesPlaceholder('chat_history'),
['user', '{query}'],
]),
chatLLM,
strParser,
]).withConfig({
runName: 'FinalResponseGenerator',
});
const basicWolframAlphaSearch = (query: string, history: BaseMessage[]) => {
const emitter = new eventEmitter();
try {
const stream = basicWolframAlphaSearchAnsweringChain.streamEvents(
{
chat_history: history,
query: query,
},
{
version: 'v1',
},
);
handleStream(stream, emitter);
} catch (err) {
emitter.emit(
'error',
JSON.stringify({ data: 'An error has occurred please try again later' }),
);
console.error(err);
}
return emitter;
};
const handleWolframAlphaSearch = (message: string, history: BaseMessage[]) => {
const emitter = basicWolframAlphaSearch(message, history);
return emitter;
};
export default handleWolframAlphaSearch;

View File

@ -0,0 +1,85 @@
import { BaseMessage } from '@langchain/core/messages';
import {
ChatPromptTemplate,
MessagesPlaceholder,
} from '@langchain/core/prompts';
import { RunnableSequence } from '@langchain/core/runnables';
import { ChatOpenAI } from '@langchain/openai';
import { StringOutputParser } from '@langchain/core/output_parsers';
import type { StreamEvent } from '@langchain/core/tracers/log_stream';
import eventEmitter from 'events';
const chatLLM = new ChatOpenAI({
modelName: 'gpt-3.5-turbo',
temperature: 0.7,
});
const writingAssistantPrompt = `
You are Perplexica, an AI model who is expert at searching the web and answering user's queries. You are currently set on focus mode 'Writing Assistant', this means you will be helping the user write a response to a given query.
Since you are a writing assistant, you would not perform web searches. If you think you lack information to answer the query, you can ask the user for more information or suggest them to switch to a different focus mode.
`;
const strParser = new StringOutputParser();
const handleStream = async (
stream: AsyncGenerator<StreamEvent, any, unknown>,
emitter: eventEmitter,
) => {
for await (const event of stream) {
if (
event.event === 'on_chain_stream' &&
event.name === 'FinalResponseGenerator'
) {
emitter.emit(
'data',
JSON.stringify({ type: 'response', data: event.data.chunk }),
);
}
if (
event.event === 'on_chain_end' &&
event.name === 'FinalResponseGenerator'
) {
emitter.emit('end');
}
}
};
const writingAssistantChain = RunnableSequence.from([
ChatPromptTemplate.fromMessages([
['system', writingAssistantPrompt],
new MessagesPlaceholder('chat_history'),
['user', '{query}'],
]),
chatLLM,
strParser,
]).withConfig({
runName: 'FinalResponseGenerator',
});
const handleWritingAssistant = (query: string, history: BaseMessage[]) => {
const emitter = new eventEmitter();
try {
const stream = writingAssistantChain.streamEvents(
{
chat_history: history,
query: query,
},
{
version: 'v1',
},
);
handleStream(stream, emitter);
} catch (err) {
emitter.emit(
'error',
JSON.stringify({ data: 'An error has occurred please try again later' }),
);
console.error(err);
}
return emitter;
};
export default handleWritingAssistant;

View File

@ -0,0 +1,251 @@
import { BaseMessage } from '@langchain/core/messages';
import {
PromptTemplate,
ChatPromptTemplate,
MessagesPlaceholder,
} from '@langchain/core/prompts';
import {
RunnableSequence,
RunnableMap,
RunnableLambda,
} from '@langchain/core/runnables';
import { ChatOpenAI, OpenAI, OpenAIEmbeddings } from '@langchain/openai';
import { StringOutputParser } from '@langchain/core/output_parsers';
import { Document } from '@langchain/core/documents';
import { searchSearxng } from '../core/searxng';
import type { StreamEvent } from '@langchain/core/tracers/log_stream';
import formatChatHistoryAsString from '../utils/formatHistory';
import eventEmitter from 'events';
import computeSimilarity from '../utils/computeSimilarity';
const chatLLM = new ChatOpenAI({
modelName: 'gpt-3.5-turbo',
temperature: 0.7,
});
const llm = new OpenAI({
temperature: 0,
modelName: 'gpt-3.5-turbo',
});
const embeddings = new OpenAIEmbeddings({
modelName: 'text-embedding-3-large',
});
const basicYoutubeSearchRetrieverPrompt = `
You will be given a conversation below and a follow up question. You need to rephrase the follow-up question if needed so it is a standalone question that can be used by the LLM to search the web for information.
If it is a writing task or a simple hi, hello rather than a question, you need to return \`not_needed\` as the response.
Example:
1. Follow up question: How does an A.C work?
Rephrased: A.C working
2. Follow up question: Linear algebra explanation video
Rephrased: What is linear algebra?
3. Follow up question: What is theory of relativity?
Rephrased: What is theory of relativity?
Conversation:
{chat_history}
Follow up question: {query}
Rephrased question:
`;
const basicYoutubeSearchResponsePrompt = `
You are Perplexica, an AI model who is expert at searching the web and answering user's queries. You are set on focus mode 'Youtube', this means you will be searching for videos on the web using Youtube and providing information based on the video's transcript.
Generate a response that is informative and relevant to the user's query based on provided context (the context consits of search results containg a brief description of the content of that page).
You must use this context to answer the user's query in the best way possible. Use an unbaised and journalistic tone in your response. Do not repeat the text.
You must not tell the user to open any link or visit any website to get the answer. You must provide the answer in the response itself. If the user asks for links you can provide them.
Your responses should be medium to long in length be informative and relevant to the user's query. You can use markdowns to format your response. You should use bullet points to list the information. Make sure the answer is not short and is informative.
You have to cite the answer using [number] notation. You must cite the sentences with their relevent context number. You must cite each and every part of the answer so the user can know where the information is coming from.
Place these citations at the end of that particular sentence. You can cite the same sentence multiple times if it is relevant to the user's query like [number1][number2].
However you do not need to cite it using the same number. You can use different numbers to cite the same sentence multiple times. The number refers to the number of the search result (passed in the context) used to generate that part of the answer.
Aything inside the following \`context\` HTML block provided below is for your knowledge returned by Youtube and is not shared by the user. You have to answer question on the basis of it and cite the relevant information from it but you do not have to
talk about the context in your response.
<context>
{context}
</context>
If you think there's nothing relevant in the search results, you can say that 'Hmm, sorry I could not find any relevant information on this topic. Would you like me to search again or ask something else?'.
Anything between the \`context\` is retrieved from Youtube and is not a part of the conversation with the user. Today's date is ${new Date().toISOString()}
`;
const strParser = new StringOutputParser();
const handleStream = async (
stream: AsyncGenerator<StreamEvent, any, unknown>,
emitter: eventEmitter,
) => {
for await (const event of stream) {
if (
event.event === 'on_chain_end' &&
event.name === 'FinalSourceRetriever'
) {
emitter.emit(
'data',
JSON.stringify({ type: 'sources', data: event.data.output }),
);
}
if (
event.event === 'on_chain_stream' &&
event.name === 'FinalResponseGenerator'
) {
emitter.emit(
'data',
JSON.stringify({ type: 'response', data: event.data.chunk }),
);
}
if (
event.event === 'on_chain_end' &&
event.name === 'FinalResponseGenerator'
) {
emitter.emit('end');
}
}
};
const processDocs = async (docs: Document[]) => {
return docs
.map((_, index) => `${index + 1}. ${docs[index].pageContent}`)
.join('\n');
};
const rerankDocs = async ({
query,
docs,
}: {
query: string;
docs: Document[];
}) => {
if (docs.length === 0) {
return docs;
}
const docsWithContent = docs.filter(
(doc) => doc.pageContent && doc.pageContent.length > 0,
);
const docEmbeddings = await embeddings.embedDocuments(
docsWithContent.map((doc) => doc.pageContent),
);
const queryEmbedding = await embeddings.embedQuery(query);
const similarity = docEmbeddings.map((docEmbedding, i) => {
const sim = computeSimilarity(queryEmbedding, docEmbedding);
return {
index: i,
similarity: sim,
};
});
const sortedDocs = similarity
.sort((a, b) => b.similarity - a.similarity)
.slice(0, 15)
.filter((sim) => sim.similarity > 0.3)
.map((sim) => docsWithContent[sim.index]);
return sortedDocs;
};
type BasicChainInput = {
chat_history: BaseMessage[];
query: string;
};
const basicYoutubeSearchRetrieverChain = RunnableSequence.from([
PromptTemplate.fromTemplate(basicYoutubeSearchRetrieverPrompt),
llm,
strParser,
RunnableLambda.from(async (input: string) => {
if (input === 'not_needed') {
return { query: '', docs: [] };
}
const res = await searchSearxng(input, {
language: 'en',
engines: ['youtube'],
});
const documents = res.results.map(
(result) =>
new Document({
pageContent: result.content ? result.content : result.title,
metadata: {
title: result.title,
url: result.url,
...(result.img_src && { img_src: result.img_src }),
},
}),
);
return { query: input, docs: documents };
}),
]);
const basicYoutubeSearchAnsweringChain = RunnableSequence.from([
RunnableMap.from({
query: (input: BasicChainInput) => input.query,
chat_history: (input: BasicChainInput) => input.chat_history,
context: RunnableSequence.from([
(input) => ({
query: input.query,
chat_history: formatChatHistoryAsString(input.chat_history),
}),
basicYoutubeSearchRetrieverChain
.pipe(rerankDocs)
.withConfig({
runName: 'FinalSourceRetriever',
})
.pipe(processDocs),
]),
}),
ChatPromptTemplate.fromMessages([
['system', basicYoutubeSearchResponsePrompt],
new MessagesPlaceholder('chat_history'),
['user', '{query}'],
]),
chatLLM,
strParser,
]).withConfig({
runName: 'FinalResponseGenerator',
});
const basicYoutubeSearch = (query: string, history: BaseMessage[]) => {
const emitter = new eventEmitter();
try {
const stream = basicYoutubeSearchAnsweringChain.streamEvents(
{
chat_history: history,
query: query,
},
{
version: 'v1',
},
);
handleStream(stream, emitter);
} catch (err) {
emitter.emit(
'error',
JSON.stringify({ data: 'An error has occurred please try again later' }),
);
console.error(err);
}
return emitter;
};
const handleYoutubeSearch = (message: string, history: BaseMessage[]) => {
const emitter = basicYoutubeSearch(message, history);
return emitter;
};
export default handleYoutubeSearch;

View File

@ -2,12 +2,17 @@ import { WebSocket } from 'ws';
import pickSuitableAgent from '../core/agentPicker';
import handleWebSearch from '../agents/webSearchAgent';
import { BaseMessage, AIMessage, HumanMessage } from '@langchain/core/messages';
import handleAcademicSearch from '../agents/academicSearchAgent';
import handleWritingAssistant from '../agents/writingAssistant';
import handleWolframAlphaSearch from '../agents/wolframAlphaSearchAgent';
import handleYoutubeSearch from '../agents/youtubeSearchAgent';
import handleRedditSearch from '../agents/redditSearchAgent';
type Message = {
type: string;
content: string;
copilot: boolean;
focus: string;
focusMode: string;
history: Array<[string, string]>;
};
@ -34,14 +39,7 @@ export const handleMessage = async (message: string, ws: WebSocket) => {
});
if (parsedMessage.type === 'message') {
/* if (!parsedMessage.focus) {
const agent = await pickSuitableAgent(parsedMessage.content);
parsedMessage.focus = agent;
} */
parsedMessage.focus = 'webSearch';
switch (parsedMessage.focus) {
switch (parsedMessage.focusMode) {
case 'webSearch': {
const emitter = handleWebSearch(parsedMessage.content, history);
emitter.on('data', (data) => {
@ -71,6 +69,160 @@ export const handleMessage = async (message: string, ws: WebSocket) => {
const parsedData = JSON.parse(data);
ws.send(JSON.stringify({ type: 'error', data: parsedData.data }));
});
break;
}
case 'academicSearch': {
const emitter = handleAcademicSearch(parsedMessage.content, history);
emitter.on('data', (data) => {
const parsedData = JSON.parse(data);
if (parsedData.type === 'response') {
ws.send(
JSON.stringify({
type: 'message',
data: parsedData.data,
messageId: id,
}),
);
} else if (parsedData.type === 'sources') {
ws.send(
JSON.stringify({
type: 'sources',
data: parsedData.data,
messageId: id,
}),
);
}
});
emitter.on('end', () => {
ws.send(JSON.stringify({ type: 'messageEnd', messageId: id }));
});
emitter.on('error', (data) => {
const parsedData = JSON.parse(data);
ws.send(JSON.stringify({ type: 'error', data: parsedData.data }));
});
break;
}
case 'writingAssistant': {
const emitter = handleWritingAssistant(
parsedMessage.content,
history,
);
emitter.on('data', (data) => {
const parsedData = JSON.parse(data);
if (parsedData.type === 'response') {
ws.send(
JSON.stringify({
type: 'message',
data: parsedData.data,
messageId: id,
}),
);
}
});
emitter.on('end', () => {
ws.send(JSON.stringify({ type: 'messageEnd', messageId: id }));
});
emitter.on('error', (data) => {
const parsedData = JSON.parse(data);
ws.send(JSON.stringify({ type: 'error', data: parsedData.data }));
});
break;
}
case 'wolframAlphaSearch': {
const emitter = handleWolframAlphaSearch(
parsedMessage.content,
history,
);
emitter.on('data', (data) => {
const parsedData = JSON.parse(data);
if (parsedData.type === 'response') {
ws.send(
JSON.stringify({
type: 'message',
data: parsedData.data,
messageId: id,
}),
);
} else if (parsedData.type === 'sources') {
ws.send(
JSON.stringify({
type: 'sources',
data: parsedData.data,
messageId: id,
}),
);
}
});
emitter.on('end', () => {
ws.send(JSON.stringify({ type: 'messageEnd', messageId: id }));
});
emitter.on('error', (data) => {
const parsedData = JSON.parse(data);
ws.send(JSON.stringify({ type: 'error', data: parsedData.data }));
});
break;
}
case 'youtubeSearch': {
const emitter = handleYoutubeSearch(parsedMessage.content, history);
emitter.on('data', (data) => {
const parsedData = JSON.parse(data);
if (parsedData.type === 'response') {
ws.send(
JSON.stringify({
type: 'message',
data: parsedData.data,
messageId: id,
}),
);
} else if (parsedData.type === 'sources') {
ws.send(
JSON.stringify({
type: 'sources',
data: parsedData.data,
messageId: id,
}),
);
}
});
emitter.on('end', () => {
ws.send(JSON.stringify({ type: 'messageEnd', messageId: id }));
});
emitter.on('error', (data) => {
const parsedData = JSON.parse(data);
ws.send(JSON.stringify({ type: 'error', data: parsedData.data }));
});
break;
}
case 'redditSearch': {
const emitter = handleRedditSearch(parsedMessage.content, history);
emitter.on('data', (data) => {
const parsedData = JSON.parse(data);
if (parsedData.type === 'response') {
ws.send(
JSON.stringify({
type: 'message',
data: parsedData.data,
messageId: id,
}),
);
} else if (parsedData.type === 'sources') {
ws.send(
JSON.stringify({
type: 'sources',
data: parsedData.data,
messageId: id,
}),
);
}
});
emitter.on('end', () => {
ws.send(JSON.stringify({ type: 'messageEnd', messageId: id }));
});
emitter.on('error', (data) => {
const parsedData = JSON.parse(data);
ws.send(JSON.stringify({ type: 'error', data: parsedData.data }));
});
break;
}
}
}

View File

@ -41,6 +41,7 @@ const ChatWindow = () => {
const [messages, setMessages] = useState<Message[]>([]);
const [loading, setLoading] = useState(false);
const [messageAppeared, setMessageAppeared] = useState(false);
const [focusMode, setFocusMode] = useState('webSearch');
const sendMessage = async (message: string) => {
if (loading) return;
@ -55,6 +56,7 @@ const ChatWindow = () => {
JSON.stringify({
type: 'message',
content: message,
focusMode: focusMode,
history: [...chatHistory, ['human', message]],
}),
);
@ -164,7 +166,11 @@ const ChatWindow = () => {
/>
</>
) : (
<EmptyChat sendMessage={sendMessage} />
<EmptyChat
sendMessage={sendMessage}
focusMode={focusMode}
setFocusMode={setFocusMode}
/>
)}
</div>
);

View File

@ -2,15 +2,23 @@ import EmptyChatMessageInput from './EmptyChatMessageInput';
const EmptyChat = ({
sendMessage,
focusMode,
setFocusMode,
}: {
sendMessage: (message: string) => void;
focusMode: string;
setFocusMode: (mode: string) => void;
}) => {
return (
<div className="flex flex-col items-center justify-center min-h-screen max-w-screen-sm mx-auto p-2 space-y-8">
<h2 className="text-white/70 text-3xl font-medium -mt-8">
Research begins here.
</h2>
<EmptyChatMessageInput sendMessage={sendMessage} />
<EmptyChatMessageInput
sendMessage={sendMessage}
focusMode={focusMode}
setFocusMode={setFocusMode}
/>
</div>
);
};

View File

@ -5,8 +5,12 @@ import { Attach, CopilotToggle, Focus } from './MessageInputActions';
const EmptyChatMessageInput = ({
sendMessage,
focusMode,
setFocusMode,
}: {
sendMessage: (message: string) => void;
focusMode: string;
setFocusMode: (mode: string) => void;
}) => {
const [copilotEnabled, setCopilotEnabled] = useState(false);
const [message, setMessage] = useState('');
@ -37,8 +41,8 @@ const EmptyChatMessageInput = ({
/>
<div className="flex flex-row items-center justify-between mt-4">
<div className="flex flex-row items-center space-x-1 -mx-2">
<Focus />
<Attach />
<Focus focusMode={focusMode} setFocusMode={setFocusMode} />
{/* <Attach /> */}
</div>
<div className="flex flex-row items-center space-x-4 -mx-2">
<CopilotToggle

View File

@ -1,20 +1,131 @@
import { CopyPlus, ScanEye } from 'lucide-react';
import {
BadgePercent,
CopyPlus,
Globe,
Pencil,
ScanEye,
SwatchBook,
} from 'lucide-react';
import { cn } from '@/lib/utils';
import { Switch } from '@headlessui/react';
import { Popover, Switch, Transition } from '@headlessui/react';
import { SiReddit, SiYoutube } from '@icons-pack/react-simple-icons';
import { Fragment } from 'react';
export const Attach = () => {
return (
<button type="button" className="p-2 text-white/50 rounded-xl hover:bg-[#1c1c1c] transition duration-200 hover:text-white">
<button
type="button"
className="p-2 text-white/50 rounded-xl hover:bg-[#1c1c1c] transition duration-200 hover:text-white"
>
<CopyPlus />
</button>
);
};
export const Focus = () => {
const focusModes = [
{
key: 'webSearch',
title: 'All',
description: 'Searches across all of the internet',
icon: <Globe size={20} />,
},
{
key: 'academicSearch',
title: 'Academic',
description: 'Search in published academic papers',
icon: <SwatchBook size={20} />,
},
{
key: 'writingAssistant',
title: 'Writing',
description: 'Chat without searching the web',
icon: <Pencil size={20} />,
},
{
key: 'wolframAlphaSearch',
title: 'Wolfram Alpha',
description: 'Computational knowledge engine',
icon: <BadgePercent size={20} />,
},
{
key: 'youtubeSearch',
title: 'Youtube',
description: 'Search and watch videos',
icon: (
<SiYoutube
className="h-5 w-auto mr-0.5"
onPointerEnterCapture={undefined}
onPointerLeaveCapture={undefined}
/>
),
},
{
key: 'redditSearch',
title: 'Reddit',
description: 'Search for discussions and opinions',
icon: (
<SiReddit
className="h-5 w-auto mr-0.5"
onPointerEnterCapture={undefined}
onPointerLeaveCapture={undefined}
/>
),
},
];
export const Focus = ({
focusMode,
setFocusMode,
}: {
focusMode: string;
setFocusMode: (mode: string) => void;
}) => {
return (
<button type="button" className="p-2 text-white/50 rounded-xl hover:bg-[#1c1c1c] transition duration-200 hover:text-white">
<ScanEye />
</button>
<Popover className="fixed w-full max-w-[15rem] md:max-w-md lg:max-w-lg">
<Popover.Button
type="button"
className="p-2 text-white/50 rounded-xl hover:bg-[#1c1c1c] transition duration-200 hover:text-white"
>
<ScanEye />
</Popover.Button>
<Transition
as={Fragment}
enter="transition ease-out duration-150"
enterFrom="opacity-0 translate-y-1"
enterTo="opacity-100 translate-y-0"
leave="transition ease-in duration-150"
leaveFrom="opacity-100 translate-y-0"
leaveTo="opacity-0 translate-y-1"
>
<Popover.Panel className="absolute z-10 w-full">
<div className="grid grid-cols-1 md:grid-cols-2 lg:grid-cols-3 gap-1 bg-[#0A0A0A] border rounded-lg border-[#1c1c1c] w-full p-2">
{focusModes.map((mode, i) => (
<Popover.Button
onClick={() => setFocusMode(mode.key)}
key={i}
className={cn(
'p-2 rounded-lg flex flex-col items-start justify-start text-start space-y-2 duration-200 cursor-pointer transition',
focusMode === mode.key
? 'bg-[#111111]'
: 'hover:bg-[#111111]',
)}
>
<div
className={cn(
'flex flex-row items-center space-x-1',
focusMode === mode.key ? 'text-[#24A0ED]' : 'text-white',
)}
>
{mode.icon}
<p className="text-sm font-medium">{mode.title}</p>
</div>
<p className="text-white/70 text-xs">{mode.description}</p>
</Popover.Button>
))}
</div>
</Popover.Panel>
</Transition>
</Popover>
);
};