Disclosure: I run this, it's Equibles. In May I posted the self-hosted version of the pipeline here. This is the hosted one, which now also covers non-US stocks and is built to be used by AI agents (over MCP) as well as by code (REST).

What's in it:

• SEC filings, full text (10-K, 10-Q, 8-K, proxies). You can search inside a filing and pull the exact lines, not just metadata
• financial statements and KPIs as reported, each tagged with fiscal period and filing date
• earnings call transcripts, by speaker
• 13F institutional holdings: holders of a stock, or a manager's full portfolio
• insider transactions (Forms 4 and 5) and congressional trades (House and Senate)
• short interest, daily short volume, fails to deliver, off-exchange volume, a squeeze score, and a model estimate of the next FINRA short interest
• executive compensation, government contracts, buybacks, going-concern flags, IPO filings
• prices for US, European, UK, Hong Kong and Chinese listings (end of day on the free plan), option chains on paid plans
• Fed economic series and CFTC data

How to get at it:

• MCP, for AI agents: https://mcp.equibles.com/mcp (Claude, ChatGPT, Cursor and others)
• REST: https://equibles.com/docs/api
• free plan: 100 requests a day across both, no card. Plus is $9.99 a month for 10,000 a day. Pro adds bulk CSV downloads (short interest, short volume, FTD, buybacks and a few more, one file per year)
• self-host: the open source core (AGPL-3.0) pulls the SEC data itself: https://github.com/daniel3303/Equibles

One detail for the backtesting crowd: short interest bulk rows carry FINRA's publication date, so you only use numbers that were public at the time.

Why "Financial Datasets alternative" in the title: it's the name I see most for financial data APIs aimed at AI agents. From their pricing page they don't have a free tier, and transcripts, congressional trades and short interest aren't on their list. If I've got that wrong, correct me and I'll fix the post.

So that's the pitch, a free financial data API for AI agents and the humans who build them. What would make it more useful as a dataset source for you: bulk files for more datasets, point-in-time snapshots, something else?

Looking to connect with individuals currently working in hands-on industries to collect first-person, point-of-view (POV) video data. These recordings will be used directly to train advanced AI and robotics systems.

Eligible Occupations & Tasks
We are looking for people who can record real, purposeful tasks during their regular working shifts, including:

• Event venues: stage setup and teardown, cable management, sound equipment
• Retail and grocery stores: restocking shelves, price tagging, collecting carts, receiving stock
• Construction sites: drywall, painting, masonry, tiling, roofing, cleanup
• Farms, gardens, and greenhouses: picking, planting, collecting eggs, feeding animals, irrigation
• Salons and spas: hair, nails, pet grooming
• Plumbing, air conditioning, and electrical work: installing faucets and toilets, servicing AC units, fitting pipes, wiring outlets and lights
• Hotels: cleaning rooms, deep cleaning bathrooms, laundry, moving luggage
• Auto shops and car washes: oil changes, tire service, detailing, motorcycle repair
• Carpentry and woodworking shops: furniture assembly, sanding, finishing, installing hardware
• Restaurants and commercial kitchens: food prep, line cooking, dishwashing, baking, cake decorating
• Commercial cleaning: restrooms, floors, windows, carpets, industrial laundry
• Food factories, warehouses, and manufacturing plants: packing, order picking, loading pallets, running machines, welding

Strict Device Requirements
To maintain data consistency, you must own one of the following high-end smartphones:

• iPhone: 12 or newer (Note: iPhone 16e and 17e are not accepted)
• Google Pixel: 6, 7, 8, or 9 (including Pro models)
• Samsung Galaxy: S21, S22, S23, or S24 (any model)

Eligible Countries
We are actively hiring and accepting data only from the following locations:

Argentina, Australia, Belize, Bolivia, Brazil, Canada, Chile, Colombia, Costa Rica, Ecuador, El Salvador, Guatemala, Honduras, Japan, Mexico, New Zealand, Nicaragua, Panama, Paraguay, Peru, Puerto Rico, Qatar, Saudi Arabia, Singapore, South Africa, South Korea, Taiwan, Ukraine, United Arab Emirates, Uruguay, Venezuela.
(More countries may become available starting in November).

If you meet the phone and country criteria and want to participate, let's talk.

Please DM if interested.

(Note to Moderators: Please feel free to remove this post if it does not fully align with the community guidelines, or let me know if I need to make any adjustments!)

I'm building a streaming price comparison and need monthly subscription prices per plan, in local currency, for ~60-100 countries. Services: Netflix, Disney+, Max, Prime Video, Apple TV+, Hulu, Peacock, Paramount+, Crunchyroll, plus regional ones like Viaplay and SkyShowtime.

What I've found so far:
- tompec/netflix-prices on GitHub: great, but Netflix only
- Global App Store Price Index (OpenTheRank): 18 countries, one plan per service, App Store prices
- Subscription Land / geopriced: good coverage for a few services, but no API or reuse licence
- TMDB/JustWatch API: availability only, no subscription prices

Is there an open or licensable dataset/API that covers more services and countries? Or is scraping each service's own pricing pages the only realistic option?

Paid sources are fine too if the licence allows showing prices on a public site.

FDA publishes pet food recalls only through its Data Dashboard and Enforcement Report search. There's no bulk download for pet food, and openFDA's recall data barely covers it. So I built one.

• recalls.csv: one row per recall, with dates (initiated, classified, terminated), class, firm, reason category and species
• products.csv: one row per product, with description, UPCs, lot codes, recall number and quantity
• excluded.csv: every veterinary row I filtered out (drugs, livestock feed…) with the reason, so you can check the filter

One finding: the median time from a company starting a recall to FDA classifying it dropped from 139 days (2012–2019) to 30 days (2020 on), and it dropped in every class. Recalls started in 2020–2023 have all had years to be classified, and their median is 28 days, so the drop isn't just recent recalls still waiting.

Page + docs: https://perbowl.com/recalls/dataset/

Also on Kaggle (with a starter notebook), GitHub and Zenodo (DOI 10.5281/zenodo.23255873). It's updated monthly.

Corrections welcome.

I was looking for egocentric video data that could be used to train robotics/physical AI models.

Interests include real-world tasks such as cooking, cleaning, picking up objects, and other everyday chores.

I was wondering what types of egocentric datasets your companies have, how many hours of footage you have, and what pricing looks like.

Hi everyone!

I'm a Master's student in Statistics at the University of Padua (Italy), currently developing my thesis on the application of Survival Analysis and Machine Learning to mortgage credit risk.

I'm looking for a European residential mortgage dataset with longitudinal loan-level observations, ideally containing:

- Loan characteristics (LTV, interest rate, loan amount, maturity, etc.).

- Repayment or performance histories.

- Default events and their timing.

- Early repayment/prepayment events, if available.

- Enough information to reconstruct loan durations and right-censoring.

My goal is to compare traditional survival models (such as Cox proportional hazards) with Survival Machine Learning methods, potentially including competing risks for default and prepayment.

I'm aware of the Freddie Mac Single-Family Loan-Level Dataset, but I would particularly like to explore European mortgage markets.

So far, I've investigated European DataWarehouse, De Nederlandsche Bank, and several academic studies using proprietary banking datasets. I've also contacted some institutions to enquire about academic access.

Does anyone know of other European mortgage datasets, research repositories, or academic collaborations that might be worth exploring?

I'm also interested in datasets that aren't publicly downloadable but might be accessible to university researchers under specific agreements.

Any suggestions, relevant papers, or contacts would be greatly appreciated.

Thanks in advance!

I'm helping out with a community research project trying to preserve and study the behavior of older GPT-3.5-Turbo models, particularly 0613 and 16k.

I'm trying to find any existing data from when these models were available, especially:

1)prompt/response datasets or cached model outputs

2)old benchmark runs with individual predictions

3)API logs containing model responses

4)token-level logprobs or top-token alternatives, if available

5)public datasets, repositories, or papers that might point us to any of the above

Even a small collection would be useful! We're particularly interested in data where the model version and collection method are documented, so we can distinguish verified historical outputs from data collected through other providers or newer models.

If you've saved anything like this, or know of a repository or person I should look into, I'd really appreciate a pointer.

Thanks :-))

I'm researching phishing detection that combines email header evidence with URL evidence. The public datasets I've found are either old (Enron, SpamAssassin, Nazario), synthetic (LLM-generated), malicious-only, or text-only with no headers

Does anyone know of a recent public dataset of real legitimate and phishing emails with full headers (.eml or raw)? Datasets that also include QR codes in the emails would be great, but that's optional. If one exists with a license that allows research use, please share it.

Thanks!

China publishes GDP twice over: one national figure, and a separate figure for each of its 31 provincial-level units. Added up, the 31 do not match the national total, and the sign of the gap flipped in 2019.

The columns in the CSV

• provincial_sum_trn - my addition of the 31 provincial GDP values as printed in the China Statistical Yearbook regional table (table 3-9, one edition per data year)
• national_prelim_trn - the national figure from that year's GDP communique
• national_rev_trn - the latest revision of the same year (routine verification, or the fourth/fifth economic census)
• gap_vs_prelim_pct - the difference between the first two, in percent

Ten rows, 2015-2024. Trillion yuan, current prices, as printed after conversion from 亿元. The gap is +6.81% in 2015, +1.60% in 2018, then -0.35% to -0.77% every year from 2019 on, the first year accounted under the unified regional accounting system.

Files (CC BY 4.0)

• Dataset folder, with a README on method and caveats: https://github.com/data-from-scans/posted-data/tree/main/china-gdp-provincial-sum-vs-national
• CSV (10 rows, 5 columns): https://github.com/data-from-scans/posted-data/blob/main/china-gdp-provincial-sum-vs-national/public/data/provincial-gdp-sum-vs-national.csv
• Figure, 2880 x 2672: https://github.com/data-from-scans/posted-data/blob/main/china-gdp-provincial-sum-vs-national/public/figures/provincial-gdp-sum-vs-national.png

Method, and the caveats that matter

The yearbook's regional tables are published as images, not as tables, so the 31 values for each year were read off the scan twice - an OCR pass and a separate visual read - and the two reads compared before use. The 2020 column was additionally re-summed a third time (1,012,415.02 亿元, +0.001% against the CSV).

Every row is one fixed vintage. NBS revises both levels - routine verification, and the fourth and fifth economic censuses - so a single year can carry two or three different official numbers. That is why the two national columns are both there: national_rev_trn shows how large the revision is per year, and measured against the later vintages the gap for 2019-2024 runs from -0.10% to -1.00%, with 2023 at -3.35% after the fifth economic census. Nothing here is a forecast.

Source: National Bureau of Statistics of China - the yearbook regional table (e.g. table 3-9, 2025 edition, a scan) and the annual GDP communiques. The underlying statistics are theirs, not mine.

Disclosure: I compiled and host these files (GitHub, CC BY 4.0).

So I'm working on a web scraping project for one of my uni courses. I need to make a large dataset with the headers mentioned above. Where can I find this data to webscrape? This is for American football.

We need effective-dated ticker/security identity, delistings, mergers, splits, dividend events, original announcement knowledge-time, revisions/cancellations, and preferably immutable historical versions rather than a corrected current-state table.

Free/open is ideal, but I’m also interested in datasets people have built themselves or vendors worth evaluating.

The key requirement is: what was known at time T must be reconstructable later without future corrections leaking backward.

If you’re building a data API, historical dataset, parser, or research infrastructure in this area, feel free to reply or DM. We are willing to stress-test it and return reproducible failure cases rather than generic feedback.

Indian Government has a website for fetching open data www.data.gov.in? But I am unable to get data from it. Has anyone tried.. is it working?

Since May 2024 every new model got the same prompts: draw a pelican on a bicycle as an SVG, build a landing page in one HTML file, tell five jokes, pick a favourite film, 87 in all. One attempt each, no system prompt, temp 0.7. I kept every answer, and it's now a dataset:

• JSON Lines, one file per question, plus models and questions
• SVG drawings as given, HTML pages with a 1280x800 screenshot each, image and audio answers as files
• release dates, and which of the 378 models are retired (101 aren't served on OpenRouter anymore)
• CC BY 4.0, DOI 10.5281/zenodo.23305275

GitHub: https://github.com/museumofmodels/collection Zenodo: https://doi.org/10.5281/zenodo.23305275 Browse it: https://museumofmodels.com

Nothing is scored, it's just the raw answers. Happy to hear what would make it more useful.

Disclosure: I made the museum and the dataset.

With parliamentary elections approaching in Serbia, numerous Instagram accounts critical of the ruling government have experienced sudden waves of suspicious followers, sometimes thousands within just a few hours.

We've collected around 1.3 million follower records and published them as an open dataset on Hugging Face:

https://huggingface.co/datasets/gavrilo/botijada

The dataset includes follow timestamps and pseudonymized account IDs that remain consistent across different profiles.

There are quite a few interesting directions to explore here: temporal patterns, shared followers between accounts, network structures, clustering, anomaly detection, and potentially signs of coordinated activity.

I'm sharing this mostly because I think there's a lot of room to experiment with the data, especially if you're into graph analysis, data visualization, or finding interesting patterns in messy real-world datasets.

Feel free to play around with it, build something, or share whatever you discover. I'd be genuinely interested in seeing different approaches and interpretations.

we run a small fund and we built a rudimentary system that tracks total headcount over time for the companies we watch. what we actually wanted was headcount by department over time, because total headcount moves too slowly to tell you anything and department level is where the signal is.

we never put much time into it because scraping is a pain, and we do not want to become a data provider. i want the team spending its time on the analysis, not on deciding which proxy to buy this month.

so we looked at buying instead. one vendor has this pattern where you fetch ids first and then collect the data in a second call, which strains our infra and works out around 8 cents a returned company. another returns great data at roughly the same price. most of the rest dont have enough filters to be worth it. and a flat file means we host it ourselves, which is a whole feature build.

where did you draw the build versus buy line on this? and has anybody found department level headcount over time as an actual field rather than a crawl

hello
if anyone can help me access these datasets needed for my grad project https://cbonds.com/indexes/100122/, https://www.bloomberg.com/professional/products/indices/quote/GBGLTRUU:IND

I run VeritaHire, which reads job postings straight from US employers' own career sites (Workday, Greenhouse, iCIMS and others) every day, about 2.5 million live postings right now. From that I publish a daily Job Openings Index, free on GitHub:

https://github.com/veritahire/us-job-openings-index

- Daily since Sept 4, 2026 (= 100), published the next morning around 08:25 UTC

- Fixed panel: 18,310 employers, 2,081,433 open postings as of Oct 10

- Columns: daily index, 7-day average, the first-published value (every first print is kept, so revisions are visible), panel size, weekly new-posting and closing rates

- CSV, CC BY 4.0, refreshed every morning by a GitHub Action

Latest: 101.06 (7-day average) through Oct 10, +1.13% over 28 days. Method: https://veritahire.com/job-openings-index/

I need to find data sets FED Overnight Index Swap (OIS) history from 2015 till 2026. I need The current month OIS and the 1 year OIS data. Also I need the federal future funds histocrical data in the current month and 1 year.

https://en.macromicro.me/charts/115044/us-overnight-indexed-swaps