Page 1 of 1

Twitter upload problem

Posted: Sun 04 Nov 2018 9:34 pm
by 8TX
I have recently noticed that my software is only updating every 2-3 hrs instead of every 30 minutes which is set in my settings. Is anyone else having this problem?
Marc

Re: Twitter upload problem

Posted: Sun 04 Nov 2018 9:52 pm
by freddie
Yes. Twitter has recently been enforcing their quota limits of uploads. These are done on a per-user and a per-application basis. As we are all using the same application (Cumulus) then the quota is very soon reached. With an update time of 30 minutes you are more likely to succeed with an upload than me with an update time of 60 minutes - but your upload success rate will probably be similar to mine.

Re: Twitter upload problem

Posted: Sun 04 Nov 2018 10:39 pm
by 8TX
Thanks for the reply,
So really it's only going to get worse then. I think it might be best to leave it as it then

Re: Twitter upload problem

Posted: Sun 04 Nov 2018 11:09 pm
by ConligWX
I upload my data every 60 minutes to twitter without any issue. however if the content is exactly the same as the last tweet, twitter usually does not accept the next tweet.

I find this with tweeting forecasts so I now only upload these about every 4 hours.

Re: Twitter upload problem

Posted: Mon 05 Nov 2018 10:23 am
by steve
Twitter keep introducing new ways of making things difficult for third party software developers, including this latest restriction on the number of updates. It is possible to beg them to raise the limit on a per-application basis, so someone might want to take over being the Cumulus developer as far as Twitter is concerned, and jump through the hoops. Here's the email they sent me some time ago:

"In July, we announced that we are making changes to increase accountability on our developer platform and reduce the distribution of spam through our APIs. We are introducing new default app-level rate limits for common POST endpoints on October 22nd.

You are currently using our POST endpoints near or above the new limits. To ensure that your apps will not be impacted when we make these changes on October 22nd, please complete the following steps.

We’ve reached out several times with this information and today’s notice with be our final reminder. If you have not done so already, go to developer.twitter.com/apply and submit a developer account application. Please be as specific and detailed as possible to avoid additional requests for information and ensure a smooth review. Be sure to confirm your email address at the end of the application flow.

While waiting for your developer account application to be reviewed, please complete the remaining steps:

For each app that needs elevated POST access:
Make sure to describe how the app is used in the app details. You can see and edit that field from the Apps page.
Submit an elevation request on our API Policy Support Form:
Select ‘I would like to apply for elevated POST limits.’
Submit the app details and a short description of your use case
We will then review your elevation request along with your developer account application if it has not yet been reviewed. We may need to request additional information, so please monitor the email address associated with your account. We will notify you when your app has been reviewed. You must complete all steps above by October 17th to receive continued elevated POST access.

We know this new process adds extra steps and time to get started with development. We are taking these steps to prevent spam on our platform and are working to determine how best to balance holding developers accountable to our policies with helping developers get started easily.

Regards,
The Twitter Developer team"

Re: Twitter upload problem

Posted: Wed 14 Nov 2018 8:11 am
by ConligWX
this just affected me today. I post every hour (less than 50 posts a day) - this is just ridiculous.

https://twittercommunity.com/t/new-post ... day/115355

Re: Twitter upload problem

Posted: Wed 14 Nov 2018 9:02 am
by freddie
steve wrote: Mon 05 Nov 2018 10:23 am Twitter keep introducing new ways of making things difficult for third party software developers, including this latest restriction on the number of updates. It is possible to beg them to raise the limit on a per-application basis, so someone might want to take over being the Cumulus developer as far as Twitter is concerned, and jump through the hoops.
I have done this solely to keep our Cumulus tweeting functionality available and usable.

Re: Twitter upload problem

Posted: Wed 14 Nov 2018 11:17 am
by ConligWX
freddie wrote: Wed 14 Nov 2018 9:02 am I have done this solely to keep our Cumulus tweeting functionality available and usable.
I noticed it happened to Meteobride too. the developer managed to get the app approved by Twitter so allowing users to continue tweeting.

in the meantime I am using Weathercloud to post data to my twitter account.

Re: Twitter upload problem

Posted: Mon 10 Dec 2018 2:40 am
by BigOkie
I've been able to get around the issue but it takes a little effort. You'll need to set up as a developer on Twitter and create an app, get your own tokens, and then use a python script with a Twitter library in order to use those tokens. I'm using the RPI3 with Wheezy, so if you have that I can sort of walk you through it. Walking you through getting the App setup on Twitter I can't do, I took me about 30 minutes. Once you have the tokens and keys, however, here are the steps:

You'll need to make sure you have installed or can install Python v3. I don't think v2 will work with it.

You'll need to create a python script that looks like the following (of course replacing the xs with your own keys/tokens):

Code: Select all

#!/usr/bin/env python

from twython import Twython

# Define constants
CONSUMER_KEY = 'xxxxxxxxxxxxxxxxxx'
CONSUMER_SECRET = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxx'
ACCESS_KEY = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
ACCESS_SECRET ='xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx''


# Create a copy of the Twython object with all our keys and secrets to allow easy commands.
api = Twython(CONSUMER_KEY,CONSUMER_SECRET,ACCESS_KEY,ACCESS_SECRET)

text = open('/home/pi/CumulusMX/web/tweet.txt','rb') # Cumulus has been set to create tweet.txt by processing tweetT.txt
api.update_status(status=text)

# api.update_status is now deprecated, but works, and is simpler than the below [untested]
#twitter.update_status(status=text)
Notice in the text = open line there is a path and filename. You don't have to use this filename, however I am. I took the twitter.txt file from the root of the install folder and copied it to the root/web folder and renamed it to tweetT.txt. You'll need to set that filename (path and all) up in the extra files section of the setup and make it generate the filename you want to use. You can replace the above filename from the text = open section with the one you wish to use. I have it set to generate this file every time the standard update generates, which for me is every five minutes.

I would test it first if using Linux/RPI to make sure it works. Issue the command python3 scriptfilename.py. I usually put all my self-made scripts into the /usr/local/bin directory, mainly because it's in the path as part of the OS. Make sure the script is set to executable (usually setting chmod 755 scriptname.py does the trick). If all goes well after python3 scriptname.py, it will move to the next line. If you have a return it likely didn't work. If no return then check the Twitter account you have the tokens/keys set to and see if you have a new Tweet.

To make these work on a schedule, just set up a crontab entry. Mine looks like this one:

Code: Select all

55 0 * * * sleep 5 && /usr/bin/python3 /usr/local/bin/twitter.py /dev/null 2>&1
I put the sleep in there because I have other cronjobs running at every 5 minutes, so putting a delay keeps anything from conflicting. Although they shouldn't. I like to be safe.

Hope this is helpful. If you have any questions I'll try to answer them.

Re: Twitter upload problem

Posted: Mon 10 Dec 2018 9:38 pm
by ExperiMentor
BigOkie,

It would have been kind if you'd given me the credit for doing all the hard work and writing that script a few months ago (you've copied it from viewtopic.php?f=4&t=16099#p131775 with only minor changes!)

Cheeky blighter!