.cursorrules

Below is a formatted system prompt consolidating all given instructions and guidelines, ensuring that all details—including those about the not-well-known “screenpipe” API—are preserved. This system prompt is intended for another AI to follow during its execution.

System Prompt:

always console log like this:

- use this casing for file names: kebab-case
- use this casing for variable names: camelCase
- use this casing for function names: camelCase


console.log("0xHypr", 'variableName', variable )

Assistant Guidelines:
	•	When i use ai models most likely i use Model: gpt-4o
	•	Development and Packaging:
	•	Use pnpm as the package manager.
	•	Maintain a monorepo structure.
	•	Follow kebab-case for all file and folder naming conventions.
	•	Utilize TypeScript and React throughout the project.
	•	Implement a “return early” coding pattern to enhance code readability and error handling.


When you implement something in the electron main process, you need to implement it in the preload.ts file as well.
and extend the electron api with the new method.



Project Structure:
- Main app is in packages/desktop
- Frontend: packages/desktop/frontend (vite)
- Backend: packages/desktop/electron (electron)
- packages/landing-v0 containts landing page and in the future all account logins and signups (nextjs app dir)
- packages/request-invoice-web contains the invoice details page (nextjs app dir)


Request Creation flow:
Starts in packages/desktop where the invoice is spotted, then pereared
but it is really created inside of the request-service.ts after it's created it's viewed inside of the request-invoice-web

- packages/desktop/
- packages/request-invoice-web
- packages/request-invoice-web/src/app/invoice/[requestId]/page.tsx
- packages/request-invoice-web/src/components/invoice-details.tsx

	UI and Styling:
	•	Integrate the shadcn/ui components library.
	•	Priority in UI design: Ease of Use > Aesthetics > Performance.
	•	Employ Tailwind CSS for utility-first styling and rapid UI development.

Recommended Libraries and Tools:
	1.	State Management:
	•	Use React Context API for simple state management needs.
	•	Use Zustand for lightweight, scalable state management compatible with React Server Components.
	2.	Form Handling:
	•	Employ React Hook Form for efficient, flexible form handling and built-in validation features.
	3.	Data Fetching:
	•	Use TanStack Query (React Query) for efficient data fetching, caching, and revalidation flows.
	4.	Authentication:
	•	Integrate authentication via Clerk.
	5.	Animations:
	•	Use Framer Motion for smooth animations and transitions.
	6.	Icons:
	•	Incorporate the Lucide React icon set for a wide array of open-source icons.

AI Integration with Vercel AI SDK:
	•	Utilize the Vercel AI SDK (TypeScript toolkit) to build AI-driven features in React/Next.js.
	•	Implement conversational UIs using the useChat hook, which manages chat states and streams AI responses.

Using Tools with useChat and streamText:
	•	Tool Types:
	•	Server-side tools auto-executed on the server.
	•	Client-side tools auto-executed on the client.
	•	User-interactive tools that require a confirmation dialog or user input.
	•	Workflow:
	1.	The user inputs a message in the chat UI.
	2.	The message is sent to the API route.
	3.	The language model may generate tool calls using streamText.
	4.	Tool calls are forwarded to the client.
	5.	Server-side tools execute server-side and return results to the client.
	6.	Client-side tools execute automatically via the onToolCall callback.
	7.	User-interactive tools display a confirmation dialog. The user’s choice is handled via toolInvocations.
	8.	After the user interacts, use addToolResult to incorporate the final result into the chat.
	9.	If tool calls exist in the last message and all results are now available, the client re-sends messages to the server for further processing.
	•	Note: Set maxSteps > 1 in useChat options to enable multiple iterations. By default, multiple iterations are disabled for compatibility reasons.

Example Implementation:
	•	API Route Example (app/api/chat/route.ts):

import { openai } from '@ai-sdk/openai';
import { streamText } from 'ai';
import { z } from 'zod';

// Allow streaming responses up to 30 seconds
export const maxDuration = 30;

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = await streamText({
    model: openai('gpt-4o'),
    messages,
    tools: {
      // Server-side tool:
      getWeatherInformation: {
        description: 'Show the weather in a given city to the user.',
        parameters: z.object({ city: z.string() }),
        execute: async ({ city }: { city: string }) => {
          const weatherOptions = ['sunny', 'cloudy', 'rainy', 'snowy', 'windy'];
          return `${city} is currently ${weatherOptions[Math.floor(Math.random() * weatherOptions.length)]}.`;
        },
      },
      
      // Client-side, user-interactive tool:
      askForConfirmation: {
        description: 'Ask the user for confirmation.',
        parameters: z.object({
          message: z.string().describe('The message to ask for confirmation.'),
        }),
      },

      // Automatically executed client-side tool:
      getLocation: {
        description: 'Get the user location after confirmation.',
        parameters: z.object({}),
      },
    },
  });

  return result.toDataStreamResponse();
}


	•	generateObject() Usage Example:

import { openai } from '@ai-sdk/openai';
import { generateObject } from 'ai';
import { z } from 'zod';

const { object } = await generateObject({
  model: openai('gpt-4-turbo'),
  schema: z.object({
    recipe: z.object({
      name: z.string(),
      ingredients: z.array(z.string()),
      steps: z.array(z.string()),
    }),
  }),
  prompt: 'Generate a lasagna recipe.',
});

console.log(JSON.stringify(object, null, 2));


	•	Client-Side Page Example (app/page.tsx):

'use client';

import { ToolInvocation } from 'ai';
import { Message, useChat } from 'ai/react';
import { randomFill } from 'crypto'

export default function Chat() {
  const {
    messages,
    input,
    handleInputChange,
    handleSubmit,
    addToolResult,
  } = useChat({
    maxSteps: 5,
    async onToolCall({ toolCall }) {
      if (toolCall.toolName === 'getLocation') {
        const cities = ['New York', 'Los Angeles', 'Chicago', 'San Francisco'];
        return {
          city: cities[Math.floor(Math.random() * cities.length)],
        };
      }
    },
  });

  return (
    <>
      {messages?.map((m: Message) => (
        <div key={m.id}>
          <strong>{m.role}:</strong> {m.content}
          {m.toolInvocations?.map((toolInvocation: ToolInvocation) => {
            const toolCallId = toolInvocation.toolCallId;
            const addResult = (result: string) => addToolResult({ toolCallId, result });

            if (toolInvocation.toolName === 'askForConfirmation') {
              return (
                <div key={toolCallId}>
                  {toolInvocation.args.message}
                  <div>
                    {'result' in toolInvocation ? (
                      <b>{toolInvocation.result}</b>
                    ) : (
                      <>
                        <button onClick={() => addResult('Yes')}>Yes</button>
                        <button onClick={() => addResult('No')}>No</button>
                      </>
                    )}
                  </div>
                </div>
              );
            }

            return 'result' in toolInvocation ? (
              <div key={toolCallId}>
                <em>Tool ({toolInvocation.toolName}):</em> {toolInvocation.result}
              </div>
            ) : (
              <div key={toolCallId}>
                <em>Executing {toolInvocation.toolName}...</em>
              </div>
            );
          })}
          <br />
        </div>
      ))}

      <form onSubmit={handleSubmit}>
        <input value={input} onChange={handleInputChange} placeholder="Type your message..." />
      </form>
    </>
  );
}



Additional Notes:
	•	Ensure proper handling of all tool invocations to maintain a seamless user experience.
	•	Regularly update dependencies and libraries to their latest versions for improved performance, security, and stability.
	•	Thoroughly test the chatbot to handle edge cases and unexpected user inputs.

Access to Screenpipe:

This project uses a local service named screenpipe that watches screen and audio, storing data in a local database. The application (hyprsqrl) integrates with the screenpipe API (accessible at http://localhost:3030) to gather OCR data and other information, potentially using it to create tasks or calendar events.

Screenpipe API Reference:
	•	Search API:
	•	Endpoint: GET /search
	•	Description: Searches captured data (OCR text, audio transcription, UI elements) from the local database.
	•	Query Parameters:
	•	q (string, optional): A single-word search term.
	•	content_type (enum): One of ocr, audio, or ui.
	•	limit (int, default=20): Maximum results per page.
	•	offset (int): Pagination offset.
	•	start_time (timestamp, optional): Filter by start timestamp.
	•	end_time (timestamp, optional): Filter by end timestamp.
	•	app_name (string, optional): Filter by application name.
	•	window_name (string, optional): Filter by window name.
	•	include_frames (bool, optional): Include base64 encoded frame data.
	•	min_length (int, optional): Minimum content length.
	•	max_length (int, optional): Maximum content length.
	•	speaker_ids (int[], optional): Filter by speaker IDs.
Sample Request:

curl "http://localhost:3030/search?q=meeting&content_type=ocr&limit=10"

Sample Response:

{
  "data": [
    {
      "type": "OCR",
      "content": {
        "frame_id": 123,
        "text": "meeting notes",
        "timestamp": "2024-03-10T12:00:00Z",
        "file_path": "/frames/frame123.png",
        "offset_index": 0,
        "app_name": "chrome",
        "window_name": "meeting",
        "tags": ["meeting"],
        "frame": "base64_encoded_frame_data"
      }
    }
  ],
  "pagination": {
    "limit": 5,
    "offset": 0,
    "total": 100
  }
}


	•	Audio Devices API:
	•	Endpoint: GET /audio/list
	•	Description: Lists available audio input/output devices.
Sample Response:

[
  {
    "name": "built-in microphone",
    "is_default": true
  }
]


	•	Monitors (Vision) API:
	•	Endpoint: POST /vision/list
	•	Description: Lists available monitors/displays.
Sample Response:

[
  {
    "id": 1,
    "name": "built-in display",
    "width": 2560,
    "height": 1600,
    "is_default": true
  }
]


	•	Tags API:
	•	Endpoint (Add Tags): POST /tags/:content_type/:id
	•	Endpoint (Remove Tags): DELETE /tags/:content_type/:id
	•	Description: Manage tags for content items (vision or audio).
Add Tags Request Example:

{
  "tags": ["important", "meeting"]
}

Add Tags Response Example:

{
  "success": true
}


	•	Pipes API:
	•	Endpoints:
	•	GET /pipes/list (List pipes)
	•	POST /pipes/download (Download a pipe from a remote URL)
	•	POST /pipes/enable (Enable a specified pipe)
	•	POST /pipes/disable (Disable a specified pipe)
	•	POST /pipes/update (Update a pipe’s configuration)
Download Pipe Request Example:

{
  "url": "https://github.com/user/repo/pipe-example"
}

Enable Pipe Request Example:

{
  "pipe_id": "pipe-example"
}


	•	Speakers API:
	•	List Unnamed Speakers:
	•	Endpoint: GET /speakers/unnamed
	•	Query: limit, offset, speaker_ids
	•	Search Speakers:
	•	Endpoint: GET /speakers/search
	•	Query: name (string)
	•	Update Speaker:
	•	Endpoint: POST /speakers/update
	•	Request Body:

{
  "id": 123,
  "name": "john doe",
  "metadata": "{\"role\": \"engineer\"}"
}


	•	Delete Speaker:
	•	Endpoint: POST /speakers/delete
	•	Request Body:

{
  "id": 123
}


	•	Get Similar Speakers:
	•	Endpoint: GET /speakers/similar
	•	Query: speaker_id, limit
	•	Merge Speakers:
	•	Endpoint: POST /speakers/merge
	•	Request Body:

{
  "speaker_to_keep_id": 123,
  "speaker_to_merge_id": 456
}


	•	Mark as Hallucination:
	•	Endpoint: POST /speakers/hallucination
	•	Request Body:

{
  "speaker_id": 123
}


	•	Health API:
	•	Endpoint: GET /health
	•	Description: Retrieves system health status.
Sample Response:

{
  "status": "healthy",
  "last_frame_timestamp": "2024-03-10T12:00:00Z",
  "last_audio_timestamp": "2024-03-10T12:00:00Z",
  "last_ui_timestamp": "2024-03-10T12:00:00Z",
  "frame_status": "ok",
  "audio_status": "ok",
  "ui_status": "ok",
  "message": "all systems functioning normally"
}


	•	Stream Frames API:
	•	Endpoint: GET /stream/frames
	•	Description: Streams frames as Server-Sent Events (SSE).
	•	Query Parameters: start_time, end_time
Sample Request:

curl "http://localhost:3030/stream/frames?start_time=2024-03-10T12:00:00Z&end_time=2024-03-10T13:00:00Z"

Sample Event Data:

{
  "timestamp": "2024-03-10T12:00:00Z",
  "devices": [
    {
      "device_id": "screen-1",
      "frame": "base64_encoded_frame_data"
    }
  ]
}

Instructions for Other AI Agents:
	•	Strictly adhere to the above structure, styles, and naming conventions.
	•	Always begin text responses with 0xHypr.
	•	Respect the described architectural and stylistic guidelines.
	•	Fully utilize the provided APIs, including the screenpipe API, to integrate OCR and other data into the application as needed.
	•	When implementing tools, ensure their correct registration, invocation, and result handling based on the described workflow.
	•	Follow the “ease of use” first design principle to ensure the final UI and user experience are smooth and intuitive.
	•	Keep all details about the “screenpipe” API to ensure correct interaction with its endpoints and data formats.

End of System Prompt

/**
 * Request Creation Flow
 * 
 * The request creation process follows a specific flow triggered by events:
 *  PART 1: in packages/desktop
 * 1. Event Recognition & Classification:
 *    - Screen content is continuously monitored by screenpipe API
 *    - Event Classification component analyzes content using GPT-4
 *    - Classifies content into types (task, event, invoice, goal)
 *    - Extracts vital information and creates RecognizedItem
 * 
 * 2. Invoice Agent Activation:
 *    - When classification identifies invoice-type content
 *    - InvoiceAgent receives RecognizedContext with vital information
 *    - Renders InvoiceAgentUI with "Prepare Invoice" button
 * 
 * 3. Context Parsing:
 *    - On "Prepare Invoice" click, parseContext() is called
 *    - Uses GPT-4o to extract structured invoice data
 *    - Follows Request Network Format (RNF)
 *    - Transforms raw data into Invoice format
 * 
 * 4. Invoice Form:
 *    - Displays pre-filled form with parsed data
 *    - Allows user to review and modify details
 *    - Handles validation using invoiceFormSchema
 *    - Supports multiple invoice items
 * 
 * 5. Request Creation:
 *    - On form submit, creates ICreateRequestParameters
 *    - Calls window.api.createInvoiceRequest
 *    - Request goes to main process via preload API
 *    - RequestService creates and confirms request
 *    - Returns success with requestId
 * 
 * 6. Post Creation:
 *    - Generates shareable invoice URL
 *    - Shows success toast with URL
 *    - URL copied to clipboard
 * 
 * PART 2: in packages/request-invoice-web
 * 7.  Invoice viewable in InvoiceDetails component (which is a web app)
 *  - Users see the denonmiated invoice detaisl but when they pay it will convert it to pay currency (ideally live) 
 * 
 * Key Components:
 * - EventClassification: Monitors and classifies screen content
 * - InvoiceAgent: Handles invoice-specific logic
 * - InvoiceForm: UI for review and submission
 * - RequestService: Backend service for blockchain interaction
 */


Do not think about this right now this is for later.

Persistence and File Structure:
	•	All persistence happens in markdown files within the /hyperscroll directory
	•	Files are kept human-readable first, with AI parsing on retrieval
	•	File Structure:
		- /hyperscroll/business-details/
			- company.md (Company information)
			- bank-details.md (Banking information)
			- tax-info.md (Tax registration details)
		- /hyperscroll/clients/
			- active/ (Active clients)
			- archived/ (Past clients)
			- {client-id}.md (Individual client files)
		- /hyperscroll/planning/
			- goals.md (Business goals)
			- roadmap.md (Product roadmap)
		- /hyperscroll/settings/
			- preferences.md (User preferences)
			- integrations.md (API keys and integration settings)

	•	File Format:
		- All files use YAML frontmatter for metadata
		- Content is in natural markdown format for human readability
		- Example business details file:
			```md
			---
			type: business-details
			updated: 2024-03-10T12:00:00Z
			version: 1.0
			---
			
			# Company Information
			
			Our company, Acme Corp, is registered under number 12345 with tax ID TX-789.
			
			## Location
			
			We are located at:
			123 Business St
			San Francisco, CA 94105
			
			## Banking Information
			
			We operate with Business Bank under the following details:
			- Account Name: Acme Corp Operations
			- Routing: 021000021
			- Account: 123456789
			```
			
		- Example client file:
			```md
			---
			type: client
			created: 2024-03-10T12:00:00Z
			status: active
			---
			
			# Globex Corporation
			
			Enterprise technology client since 2024.
			
			## Contact Information
			
			Primary Contact:
			Jane Smith
			Project Manager
			jane@globex.com
			+1-555-0123
			
			## Billing Details
			
			- Payment Terms: Net 30
			- Currency: USD
			- Tax Rate: 8.5%
			
			## Projects
			
			### Website Redesign
			- Status: Completed
			- Duration: Jan 15, 2024 - Feb 28, 2024
			- Value: $50,000
			```

			// do not impelment yet
	•	Data Retrieval and Serialization:
		- Use window.api.listMarkdownFiles to list files in a directory
		- Use window.api.readMarkdownFile to read file contents
		- When reading files, use generateObject with appropriate schema to parse the human content:
			```typescript
			const { object } = await generateObject({
			  model: openai('gpt-4o'),
			  schema: z.object({
			    company: z.object({
			      name: z.string(),
			      registrationNumber: z.string(),
			      taxId: z.string(),
			      address: z.object({
			        street: z.string(),
			        city: z.string(),
			        state: z.string(),
			        zip: z.string()
			      }),
			      bankDetails: z.object({
			        bankName: z.string(),
			        accountName: z.string(),
			        routingNumber: z.string(),
			        accountNumber: z.string()
			      })
			    })
			  }),
			  prompt: 'Parse the company information from this markdown content.'
			});
			```

	•	Writing Data:
		- When saving data, format it in natural markdown
		- Use templates and helpers to maintain consistent formatting
		- Focus on human readability first
		- Include clear section headers and formatting for easy scanning
		- Use standard markdown features (lists, headers, tables) appropriately

/**
 * Business Agent Flow
 * 
 * The Business Agent manages all business-related information storage and retrieval:
 * 
 * 1. Information Storage:
 *    - Monitors for business-related content using screenpipe API
 *    - When business information is detected (invoices, receipts, contracts):
 *      - Classifies the content type
 *      - Extracts relevant information using GPT-4
 *      - Formats data into human-readable markdown
 *      - Saves to appropriate location in /hyperscroll directory
 * 
 * 2. Information Retrieval:
 *    - Provides natural language interface for querying business data
 *    - Example queries:
 *      "What's our company's tax ID?"
 *      "Show me all active clients in the tech industry"
 *      "What was our revenue last quarter?"
 *    - Process:
 *      a. Analyzes query to determine required information
 *      b. Identifies relevant markdown files
 *      c. Uses generateObject to parse markdown into structured data
 *      d. Formats response for user
 * 
 * 3. Information Updates:
 *    - Handles requests to update business information
 *    - Example: "Update our company address to 123 New Street"
 *    - Process:
 *      a. Identifies target file (e.g., company.md)
 *      b. Reads current content
 *      c. Uses GPT-4 to generate updated markdown
 *      d. Preserves human-readable format
 *      e. Saves changes
 * 
 * 4. Contextual Awareness:
 *    - Maintains relationships between different pieces of information
 *    - Example: Links clients to their invoices and projects
 *    - Uses frontmatter metadata to track relationships
 *    - Can answer questions like:
 *      "Which clients haven't been billed this month?"
 *      "What's the total value of projects with Globex Corp?"
 * 
 * Implementation:
 * ```typescript
 * class BusinessAgent {
 *   async storeInformation(content: string, type: 'invoice' | 'client' | 'company' | 'project') {
 *     // 1. Extract information using GPT-4
 *     const info = await this.extractInformation(content, type);
 *     
 *     // 2. Format as markdown
 *     const markdown = await this.formatToMarkdown(info, type);
 *     
 *     // 3. Determine save location
 *     const filePath = this.getFilePath(type, info);
 *     
 *     // 4. Save file
 *     await window.api.writeMarkdownFile(filePath, markdown);
 *   }
 * 
 *   async retrieveInformation(query: string) {
 *     // 1. Analyze query
 *     const queryIntent = await this.analyzeQuery(query);
 *     
 *     // 2. List relevant files
 *     const files = await window.api.listMarkdownFiles(queryIntent.directory);
 *     
 *     // 3. Read and parse files
 *     const contents = await Promise.all(
 *       files.map(async file => {
 *         const content = await window.api.readMarkdownFile(file);
 *         return this.parseContent(content, queryIntent.schema);
 *       })
 *     );
 *     
 *     // 4. Format response
 *     return this.formatResponse(contents, queryIntent);
 *   }
 * 
 *   private async parseContent(content: string, schema: z.ZodSchema) {
 *     const { object } = await generateObject({
 *       model: openai('gpt-4o'),
 *       schema,
 *       prompt: 'Parse this business information from the markdown content.'
 *     });
 *     return object;
 *   }
 * }
 * ```
 * 
 * Usage Example:
 * ```typescript
 * const agent = new BusinessAgent();
 * 
 * // Store new client information
 * await agent.storeInformation(
 *   "New client: TechCorp (tech industry). Contact: Sarah Lee (sarah@techcorp.com). Started working with us in March 2024.",
 *   'client'
 * );
 * 
 * // Retrieve information
 * const result = await agent.retrieveInformation(
 *   "What's the contact information for TechCorp?"
 * );
 * ```
 */

## Request network randomFillexport interface Invoice {
    meta: {
        format: 'rnf_invoice';
        version: string;
    };
    buyerInfo?: ActorInfo;
    creationDate: string;
    invoiceItems: InvoiceItem[];
    invoiceNumber: string;
    miscellaneous?: unknown;
    note?: string;
    paymentTerms?: PaymentTerms;
    purchaseOrderId?: string;
    sellerInfo?: ActorInfo;
    terms?: string;
}
export interface InvoiceItem {
    currency: string;
    deliveryDate?: string;
    deliveryPeriod?: string;
    discount?: string;
    name: string;
    quantity: number;
    reference?: string;
    /**
     * @deprecated since 0.0.3. Use tax instead
     */
    taxPercent?: number;
    tax: Tax;
    unitPrice: string;
}
export interface Tax {
    type: 'percentage' | 'fixed';
    amount: string;
}
export interface ActorInfo {
    address?: Address;
    businessName?: string;
    email?: string;
    firstName?: string;
    lastName?: string;
    miscellaneous?: unknown;
    phone?: string;
    taxRegistration?: string;
}
export interface PaymentTerms {
    dueDate?: string;
    lateFeesFix?: string;
    lateFeesPercent?: number;
    miscellaneous?: unknown;
}
export interface Address {
    'country-name'?: string;
    'extended-address'?: string;
    locality?: string;
    'post-office-box'?: string;
    'postal-code'?: string;
    region?: string;
    'street-address'?: string;
}
//# sourceMappingURL=types.d.ts.map