Metadata Filtering and Query Optimization

Advanced filtering turns broad queries into precise results. This guide covers metadata filtering strategies, query optimization patterns, and performance techniques for production applications.

What You'll Learn

Content type and file type filtering
Date range queries
Entity-based filtering
Metadata property filters
Query optimization techniques
Batch query patterns

Part 1: Content Filtering Fundamentals

By Content Type

import { Graphlit } from 'graphlit-client';
import { ContentTypes, FileTypes } from 'graphlit-client/dist/generated/graphql-types';

const graphlit = new Graphlit();

// Only emails
const emails = await graphlit.queryContents({
  search: 'project update',
  filter: {
    types: [ContentTypes.Email]
  }
});

// Only Slack messages
const messages = await graphlit.queryContents({
  search: 'deployment',
  filter: {
    types: [ContentTypes.Message]
  }
});

// Multiple types
const communications = await graphlit.queryContents({
  search: 'budget',
  filter: {
    types: [ContentTypes.Email, ContentTypes.Message]
  }
});

By File Type

// Only PDFs
const pdfs = await graphlit.queryContents({
  search: 'annual report',
  filter: {
    types: [ContentTypes.File],
    fileTypes: [FileTypes.Document]
  }
});

// Only images
const images = await graphlit.queryContents({
  search: 'product mockup',
  filter: {
    fileTypes: [FileTypes.Image]
  }
});

// Documents and presentations
const docs = await graphlit.queryContents({
  search: 'Q4 strategy',
  filter: {
    fileTypes: [FileTypes.Document, FileTypes.Presentation]
  }
});

Part 2: Temporal Filtering

Date Range Queries

// Last 30 days
const recent = await graphlit.queryContents({
  search: 'incident report',
  filter: {
    creationDateRange: {
      from: new Date(Date.now() - 30 * 24 * 60 * 60 * 1000).toISOString()
    }
  }
});

// Specific quarter
const q4 = await graphlit.queryContents({
  search: 'financial results',
  filter: {
    creationDateRange: {
      from: '2024-10-01T00:00:00Z',
      to: '2024-12-31T23:59:59Z'
    }
  }
});

// Last 7 days
const thisWeek = await graphlit.queryContents({
  search: 'standup notes',
  filter: {
    creationDateRange: {
      from: new Date(Date.now() - 7 * 24 * 60 * 60 * 1000).toISOString()
    }
  }
});

Part 3: Entity-Based Filtering

Filter by Person

import { ObservableTypes } from 'graphlit-client/dist/generated/graphql-types';

// Find Alice's entity
const aliceEntity = await graphlit.queryObservables({
  filter: {
    searchText: "Alice Johnson",
    types: [ObservableTypes.Person]
  }
});

const aliceId = aliceEntity.observables?.results?.[0]?.observable.id;

// Content mentioning Alice
const aliceDocs = await graphlit.queryContents({
  search: '',  // Empty = all content
  filter: {
    observations: {
      observables: [{ id: aliceId }]
    }
  }
});

console.log(`Documents mentioning Alice: ${aliceDocs.contents.results.length}`);

Filter by Organization

// Find company entity
const acmeEntity = await graphlit.queryObservables({
  filter: {
    searchText: "Acme Corp",
    types: [ObservableTypes.Organization]
  }
});

const acmeId = acmeEntity.observables?.results?.[0]?.observable.id;

// Content about Acme Corp
const acmeDocs = await graphlit.queryContents({
  filter: {
    observations: {
      observables: [{ id: acmeId }]
    }
  }
});

Multi-Entity Filter (AND)

// Content mentioning BOTH Alice AND Acme Corp
const combined = await graphlit.queryContents({
  filter: {
    observations: {
      observables: [
        { id: aliceId },
        { id: acmeId }
      ]
    }
  }
});

console.log('Docs with Alice AND Acme Corp:', combined.contents.results.length);

Part 4: Combined Filters

Complex Filter Example

// Recent PDFs from Alice about Acme Corp
const complex = await graphlit.queryContents({
  search: 'partnership proposal',
  filter: {
    types: [ContentTypes.File],
    fileTypes: [FileTypes.Document],
    creationDateRange: {
      from: new Date(Date.now() - 90 * 24 * 60 * 60 * 1000).toISOString()
    },
    observations: {
      observables: [
        { id: aliceId },
        { id: acmeId }
      ]
    }
  }
});

Breakdown:

search: Semantic search for "partnership proposal"
types + fileTypes: Only PDF documents
creationDateRange: Last 90 days
observations: Mentions both Alice and Acme Corp

Part 5: Performance Optimization

Pattern 1: Pre-Filter Before Search

// Slow: Search everything, then filter in app
const all = await graphlit.queryContents({ search: 'budget' });
const pdfs = all.contents.results.filter(c => c.fileType === 'PDF');

// Fast: Filter at query time
const fast = await graphlit.queryContents({
  search: 'budget',
  filter: {
    fileTypes: [FileTypes.Document]
  }
});

Why faster: Pre-filtering reduces search space and vector scan size.

Pattern 2: Limit Results

// Get only what you need
const topResults = await graphlit.queryContents({
  search: 'machine learning',
  limit: 10  // Only top 10
});

Performance:

Top 10: ~100ms
Top 50: ~200ms
Top 100: ~500ms

Pattern 3: Pagination

// Page 1
const page1 = await graphlit.queryContents({
  search: 'AI research',
  limit: 20,
  offset: 0
});

// Page 2
const page2 = await graphlit.queryContents({
  search: 'AI research',
  limit: 20,
  offset: 20
});

Part 6: Batch Query Patterns

Parallel Queries

// Query multiple types in parallel
const [emails, messages, docs] = await Promise.all([
  graphlit.queryContents({
    search: 'quarterly update',
    filter: { types: [ContentTypes.Email] }
  }),
  graphlit.queryContents({
    search: 'quarterly update',
    filter: { types: [ContentTypes.Message] }
  }),
  graphlit.queryContents({
    search: 'quarterly update',
    filter: { types: [ContentTypes.File] }
  })
]);

console.log('Emails:', emails.contents.results.length);
console.log('Messages:', messages.contents.results.length);
console.log('Documents:', docs.contents.results.length);

Faceted Search

// Get counts by type
const allResults = await graphlit.queryContents({
  search: 'project phoenix'
});

const facets = {
  emails: allResults.contents.results.filter(c => c.type === ContentTypes.Email).length,
  messages: allResults.contents.results.filter(c => c.type === ContentTypes.Message).length,
  documents: allResults.contents.results.filter(c => c.type === ContentTypes.File).length
};

console.log('Results by type:', facets);

Part 7: Query Building Patterns

Dynamic Filter Builder

interface FilterOptions {
  contentTypes?: ContentTypes[];
  fileTypes?: FileTypes[];
  dateFrom?: Date;
  entityIds?: string[];
}

function buildFilter(options: FilterOptions) {
  const filter: any = {};
  
  if (options.contentTypes && options.contentTypes.length > 0) {
    filter.types = options.contentTypes;
  }
  
  if (options.fileTypes && options.fileTypes.length > 0) {
    filter.fileTypes = options.fileTypes;
  }
  
  if (options.dateFrom) {
    filter.creationDateRange = {
      from: options.dateFrom.toISOString()
    };
  }
  
  if (options.entityIds && options.entityIds.length > 0) {
    filter.observations = {
      observables: options.entityIds.map(id => ({ id }))
    };
  }
  
  return Object.keys(filter).length > 0 ? filter : undefined;
}

// Usage
const results = await graphlit.queryContents({
  search: 'budget',
  filter: buildFilter({
    contentTypes: [ContentTypes.Email],
    dateFrom: new Date(Date.now() - 30 * 24 * 60 * 60 * 1000),
    entityIds: [aliceId]
  })
});

Common Patterns

Pattern: Find Recent Content from Specific Person

async function findRecentFromPerson(personName: string, days: number) {
  // Find person entity
  const person = await graphlit.queryObservables({
    filter: {
      searchText: personName,
      types: [ObservableTypes.Person]
    }
  });
  
  const personId = person.observables?.results?.[0]?.observable.id;
  
  if (!personId) {
    throw new Error(`Person "${personName}" not found`);
  }
  
  // Query recent content
  return graphlit.queryContents({
    filter: {
      observations: {
        observables: [{ id: personId }]
      },
      creationDateRange: {
        from: new Date(Date.now() - days * 24 * 60 * 60 * 1000).toISOString()
      }
    }
  });
}

// Usage
const aliceRecent = await findRecentFromPerson('Alice Johnson', 30);

What's Next?

You now master metadata filtering. Apply these patterns to:

Build filtered search UIs
Optimize query performance
Create faceted search

Related guides:

Complete Guide to Search - Search basics
Knowledge Graph Queries - Entity queries
Building AI Chat Applications - Filtered RAG

Congrats! You've completed all 20 Developer Guides! 🎉