Following technical topics on Twitter is not always easy. Why?
Because:
It is not possible to follow everyone who tweets about the topic you are interested in.
Even if you do so, they not always tweet about the topics you are interested in.
You can continuously use Twitter's search option, but even their results are full of unrelated content.
Most interesting hashtags are polluted with marketing material, AKA spam!
As an example, if you search for Python as a programming language, you will get a mix of technical content and news about giant snakes around the world!
Tweets are in different languages, and you cannot easily filter out the non-English ones.
If you search for a technical topic, you don't want to get pornography or other unrelated material, but this can happen because there might be a similarity between the words you have searched for and a username on Twitter or the word might have been used in the middle of a link that someone has shared.
How to tackle the problem?
In this article I had already explained how to build a simple Twitter bot with Python and especially how to deal with authentication. The subject-based Twitter bots that I have built are also written in Python and have go through the same process for getting approvals, tokens, etc. I am also running them on AWS Lambda functions that are triggered by AWS EventBridge every six minutes.
Even though the logic would change in the future, but for now I check these conditions:
I use Amazon Comprehend to check if the content of the Tweet, and the description in the user profile are in English. If you want to use this service, don't forget that your application needs proper permissions to access it. You can either use an AWS managed policy like ComprehendReadOnly or create your fine-grained one. If you decide to use this service, make sure you check the pricing in advance and keep an eye on your AWS costs.
I use Amazon Rekognition to assure the user who has tweeted is not using nudity, violence, or hate simbols in their profile photo or the background image they have used. If you want to use this service, don't forget that your application needs proper permissions to access it. You can either use an AWS managed policy like AmazonRekognitionReadOnlyAccess or create your fine-grained one. If you decide to use this service, make sure you check the pricing in advance and keep an eye on your AWS costs, because Rekognition is one of the most expensive services that Amazon provides.
I check if the tweet itself contains any of the words related to the topic.
I check the content of the tweet to assure it doesn't contain unrelated or unwanted words.
I check user's name and description to assure they don't contain unwanted words.
I check if the tweet is a quote to another one. If so, I check the content of the quoted tweet to assure it doesn't contain unwanted words.
I check if the user who has tweeted is not too new or too unpopular. That's usually a sign of spam.
I check if the user tries to abuse the bot, for example by tweeting too much.
I check if the tweet contains too many hashtags. That's usually a sign of abusing hashtags.
I check if the tweet contains too many new-lines. That's usually a sign of spam.
These checks are not covering all the possibilities and will probably be improved in the future. I also might use the power of natural language processing by training a custom model and labeling based on some known articles.
List of the bots
Retweets contents related to Cloud infrastructure, AWS, GCP, Azure, Cloudflare, Kubernetes, Docker, etc.
Retweets contents related to Python programming language.
Retweets contents related to Golang programming language.
Retweets contents related to PHP, Laravel, Symfony, etc.
Final notes
Even though these bots are not perfect yet, but they could manage to exclude about 95% - 98% of spam, unwanted and unrelated content.
Percentage of related content vs spam
If you like the idea, don't forget to follow any of the bots from the list above. Also, if you want to be notified of my future articles, don't forget to follow me either on Twitter at @pooyan_razian. or on Medium at @pooyan_razian.
Cheers! 🍻
