Have Geek Brain

Pulling multiple pages of search queries from Twitter

I am trying to find a way to pull all tweets of a specific search term via PHP and a Twitter search query.

So the functionality would include 1. Include search query 2. Extract terms from each page. 3. Grab new terms from last search 4. Export to db or flat file.

I understand enough of all these cases, except for navigating through multiple pages

0


a source to share


3 answers


The twitter API takes a page number parameter. Atom results have link elements, with rel attributes for next and previous. This will be your best indicator of whether or not you should be looking for a second page, and so on. The href attribute of this tag will even tell you the url you should request.

The generated request also takes a from_id parameter. You will want to keep the largest ID number that you see in your responses and use that in subsequent requests so you don't have to filter out duplicates.



When it comes to storing data, your choice will probably best be guided by what you plan to do with the results ... if you are going to make any queries you should probably defer them to the database, i.e. MySQL. If you're just signing up, a flat file should do you well.

+5


a source


The search API has a parameter page

:

: Not necessary. Page number (starting at 1) to return, up to a maximum of 1500 results (based on rpp page *. Note: there are pagination limits.

Example: http://search.twitter.com/search.atom?q=devo&rpp=15&page=2



Does it help?

0


a source


twitter search api takes a page parameter:

# page: optional. Page number (starting at 1) to return, up to max. approximately 1500 results (based on rpp * page. Note: there are pagination limits.

* Example: http://search.twitter.com/search.atom?q=devo&rpp=15&page=2
0


a source






More articles:


All Articles