Python script executed via cron is not accidentally executed
I have a simple python script to fetch tweets and cache them to disk, which is configured to run every two minutes via cron.
*/2 * * * * (date ; /usr/bin/python /path/get_tweets.py) >> /path/log/get_tweets.log 2>&1
The script runs successfully most of the time. However, the script fails every time. In addition to other logging, I added a simple print statement above the meat of the script, and nothing but the output from the start date command is logged.
#!/usr/bin/python
# Script for Fetching Tweets and then storing them as an HTML snippet for inclusion using SSI
print "Starting get_tweets.py"
import simplejson as json
import urllib2
import httplib
import re
import calendar
import codecs
import os
import rfc822
from datetime import datetime
import time
import sys
import pprint
debug = True
now = datetime.today()
template = u'<p class="tweet">%s <span class="date">on %s</span></p>'
html_snippet = u''
timelineUrl = 'http://api.twitter.com/1/statuses/user_timeline.json?screen_name=gcorne&count=7'
tweetFilePath = '/path/server-generated-includes/tweets.html'
if(debug): print "[%s] Fetching tweets from %s." % (now, timelineUrl)
def getTweets():
request = urllib2.Request(timelineUrl)
opener = urllib2.build_opener()
try:
tweets = opener.open(request)
except:
print "[%s] HTTP Request %s failed." % (now, timelineUrl)
exitScript()
tweets = tweets.read()
return tweets
def exitScript():
print "[%s] Script failed." % (now)
sys.exit(0)
tweets = getTweets()
now = datetime.today()
if(debug): print "[%s] Tweets retrieved." % (now)
tweets = json.loads(tweets)
for tweet in tweets:
text = tweet['text'] + ' '
when = tweet['created_at']
when = re.match(r'(\w+\s){3}', when).group(0).rstrip()
# print GetRelativeCreatedAt(when)
# convert links
text = re.sub(r'(http://.*?)\s', r'<a href="\1">\1</a>', text).rstrip()
#convert hashtags
text = re.sub(r'#(\w+)', r'<a href="http://www.twitter.com/search/?q=%23\1">#\1</a>', text)
# convert @ replies
text = re.sub(r'@(\w+)', r'@<a href="http://www.twitter.com/\1">\1</a>', text)
html_snippet += template % (text, when) + "\n"
#print html_snippet
now = datetime.today()
if(debug): print "[%s] Opening file %s." % (now, tweetFilePath)
try:
file = codecs.open(tweetFilePath, 'w', 'utf_8')
except:
print "[%s] File %s cound not be opened." % (now, tweetFilePath)
exitScript()
now = datetime.today()
if(debug): print "[%s] Writing %s to disk." % (now, tweetFilePath)
file.write(html_snippet)
now = datetime.today()
if(debug): print "[%s] Finished writing %s to disk." % (now, tweetFilePath)
file.close()
sys.exit(0)
Any ideas? VPS system works with Centos 5.3 using python 2.4.
Update: I've added the entire script to avoid confusion.
a source to share
The most likely explanation is that the script time takes more than two minutes (maybe the system is very busy at times, or the script may have to wait for some external site that is busy sometimes, etc.) and your cron is sane which skips recurring events that have not yet completed. By logging the start and end times of your script, you should be able to double check if so. What you want to do in such circumstances is up to you (I recommend that you consider skipping an accidental startup to avoid further overloading a very busy system - your own or the remote one from which you are receiving data).
a source to share
I had a problem with a Python script that sometimes didn't run in crontab, but always ran from the command line. It turns out I had to redirect the logging to /dev/null
. Otherwise, the standard output will be full and the program will simply stop and the process will be removed. Using /dev/null
to dump the output and everything is fine.
a source to share