← All chapters18 / 30
Using Chirai · Chapter 18
Data feeds
A website can't crawl other websites by itself. Data feeds bring records from a real source into your site's own tables, on a schedule you choose. Examples: news about a topic, companies from a list, items from an RSS feed, rows from a Google Sheet.
Open Manage site → Settings → Data feeds.
Sources
- News and press releases for any words (Google News).
- Y Combinator companies (a public list, updated daily).
- Any RSS or Atom feed.
- Any JSON API. Name the list inside the answer, e.g.
data.items. - A CSV file or a published Google Sheet.
- Paid providers (Crunchbase, PitchBook and others): add your own API key in Connections, then choose it in the feed. Chirai doesn't copy their websites.
Adding a feed
- Choose the source and press Read it once. Chirai fetches it and shows its fields.
- Choose the table, and which field goes into each column. Matching names are filled in for you (title → title, link → source_url).
- Choose the column that identifies a record (e.g. slug or name), so a record is updated, not added twice.
- Choose how often: every 1, 6, 12 or 24 hours, or weekly. Then press Save and run now.
Each run adds new records and updates existing ones (up to 500 a run). Manage site shows when each feed last ran, how many records it brought, and any error.
What visitors see
A site with feeds shows a small note: "Data updated 3 h ago · 2 sources". A site whose request needs outside data, but has no feed yet, says "Sample data · live sources not connected yet". That way starter rows are never presented as real.