Recent comments in /f/technology

yukeake t1_jc1vmxy wrote

Any sort of social research that isn't extremely time-sensitive could make use of week-old data. Academia is the big one, though, as they're always cash-strapped and there's no way they could afford thousands of dollars a month.

4

tyler1128 t1_jc1vmhb wrote

I do wonder the legality of offering a side market for buying twitter data scraped from what the website freely gives you. I'm sure those "hacker" forums still sell sock proxy lists on the daily. That plus beautiful soup and not being stupid in how you do it should be both a weekend prototype level project, and pretty cheap. It's been a while since I've done something like that, but socks proxies are a dime a dozen more or less. Now, you are probably utilizing hacked servers, but you aren't hacking them so pleading ignorance would probably do just fine. Plus, Twitter is hardly capable of keeping running now, not sure their scraper detection is exactly "state of the art".

6

DevAway22314 t1_jc1scwh wrote

It sounds like the goal is to get rid of and/or discredit independent research

No one will pay it, so research will die down. Some woll use scrapers, but then he can just claim they're inaccurate because they aren't "official" results from the API

29

Westfakia t1_jc1r8lo wrote

Good for whom?

Giving preferential treatment to financial companies based on means alone isn’t going to benefit anyone except the people at the top, who will continue to work to widen the gap between them and everyone else.

2

londons_explorer t1_jc1g3ee wrote

There are many companies who would be willing to pay far more than this for the data.

Things like investment firms who want to know what is going on and react in real time.

The real solution is to have a delayed data feed - everything more than a week old is available for free. If you want data 15 mins delayed, pay $$. If you want data 1 second delayed, pay $$$$. If you want data immediately, pay $$$$$$$$.

4