Notebook

Wrangle and Analyze Data¶

Wrangling of WeRateDogs Twitter data to create interesting and trustworthy analyses and visualizations.

Table of Contents¶

Introduction
Gathering
Assessing
Cleaning
Storing Cleaned Data
Analyzing and Visualizing

Introduction¶

The WeRateDogs Twitter archive is great, but it only contains very basic tweet information. So we perform additional gathering through an URL and a direct connection to twitter. Then we assess and clean the Data as required to perform analysis and visualization.

In [784]:

# Import required packages
import pandas as pd
import numpy as np
import requests
import tweepy
import json
import re
import matplotlib.pyplot as plt
import seaborn as sns
from timeit import default_timer as timer

Gathering¶

Based on the project motivation and details, we gather Data from three sources:

The WeRateDogs twitter archive, available as twitter_archive_enhanced.csv
The tweet image predictions, i.e., what breed of dog (or other object, animal, etc.) is present in each tweet according to a neural network, available at the following URL: https://d17h27t6h515a5.cloudfront.net/topher/2017/August/599fd2ad_image-predictions/image-predictions.tsv
A pandas DataFrame created from data we will be collecting directly from twitter

1. Gather data from the archive available as `twitter-archive-enhanced.csv`¶

In [785]:

# Load the csv file as a dataframe
df_archive = pd.read_csv('twitter-archive-enhanced.csv')

In [786]:

# make a copy, as we will be using the copy
df_archive_clean = df_archive.copy()

In [787]:

# Get a view on the dataframe
df_archive_clean.head()

Out[787]:

	tweet_id	in_reply_to_status_id	in_reply_to_user_id	timestamp	source	text	retweeted_status_id	retweeted_status_user_id	retweeted_status_timestamp	expanded_urls	rating_numerator	rating_denominator	name	doggo	floofer	pupper	puppo
0	892420643555336193	NaN	NaN	2017-08-01 16:23:56 +0000	<a href="http://twitter.com/download/iphone" rel="nofollow">Twitter for iPhone</a>	This is Phineas. He's a mystical boy. Only ever appears in the hole of a donut. 13/10 https://t.co/MgUWQ76dJU	NaN	NaN	NaN	https://twitter.com/dog_rates/status/892420643555336193/photo/1	13	10	Phineas	None	None	None	None
1	892177421306343426	NaN	NaN	2017-08-01 00:17:27 +0000	<a href="http://twitter.com/download/iphone" rel="nofollow">Twitter for iPhone</a>	This is Tilly. She's just checking pup on you. Hopes you're doing ok. If not, she's available for pats, snugs, boops, the whole bit. 13/10 https://t.co/0Xxu71qeIV	NaN	NaN	NaN	https://twitter.com/dog_rates/status/892177421306343426/photo/1	13	10	Tilly	None	None	None	None
2	891815181378084864	NaN	NaN	2017-07-31 00:18:03 +0000	<a href="http://twitter.com/download/iphone" rel="nofollow">Twitter for iPhone</a>	This is Archie. He is a rare Norwegian Pouncing Corgo. Lives in the tall grass. You never know when one may strike. 12/10 https://t.co/wUnZnhtVJB	NaN	NaN	NaN	https://twitter.com/dog_rates/status/891815181378084864/photo/1	12	10	Archie	None	None	None	None
3	891689557279858688	NaN	NaN	2017-07-30 15:58:51 +0000	<a href="http://twitter.com/download/iphone" rel="nofollow">Twitter for iPhone</a>	This is Darla. She commenced a snooze mid meal. 13/10 happens to the best of us https://t.co/tD36da7qLQ	NaN	NaN	NaN	https://twitter.com/dog_rates/status/891689557279858688/photo/1	13	10	Darla	None	None	None	None
4	891327558926688256	NaN	NaN	2017-07-29 16:00:24 +0000	<a href="http://twitter.com/download/iphone" rel="nofollow">Twitter for iPhone</a>	This is Franklin. He would like you to stop calling him "cute." He is a very fierce shark and should be respected as such. 12/10 #BarkWeek https://t.co/AtUZn91f7f	NaN	NaN	NaN	https://twitter.com/dog_rates/status/891327558926688256/photo/1,https://twitter.com/dog_rates/status/891327558926688256/photo/1	12	10	Franklin	None	None	None	None

From the project details, we got the information that the data we loaded are probably not all correct: the ratings are probably not all correct, same for the dog names and dog stages.
So we will need to assess and clean those columns prior to any analysis and visualization.

2. Gather the tweet image predictions¶

As the image-predictions.tsv is available through the following URL https://d17h27t6h515a5.cloudfront.net/topher/2017/August/599fd2ad_image-predictions/image-predictions.tsv, I download it first using the "Requests" library, and then I load it as a dataframe.

In [5]:

# Set the images file URL
images_url = 'https://d17h27t6h515a5.cloudfront.net/topher/2017/August/599fd2ad_image-predictions/image-predictions.tsv'

In [6]:

# Get the content 
response = requests.get(images_url)

In [7]:

# Write it locally as a file
with open('tweet_images_predictions.tsv', mode='wb') as file:
        file.write(response.content)

In [788]:

# load the images prediction file as a dataframe
df_image = pd.read_csv('tweet_images_predictions.tsv', sep='\t')

In [789]:

# create a copy for our further cleaning
df_image_clean = df_image.copy()

In [790]:

# Get a view to this dataframe
df_image_clean.head()

Out[790]:

	tweet_id	jpg_url	img_num	p1	p1_conf	p1_dog	p2	p2_conf	p2_dog	p3	p3_conf	p3_dog
0	666020888022790149	https://pbs.twimg.com/media/CT4udn0WwAA0aMy.jpg	1	Welsh_springer_spaniel	0.465074	True	collie	0.156665	True	Shetland_sheepdog	0.061428	True
1	666029285002620928	https://pbs.twimg.com/media/CT42GRgUYAA5iDo.jpg	1	redbone	0.506826	True	miniature_pinscher	0.074192	True	Rhodesian_ridgeback	0.072010	True
2	666033412701032449	https://pbs.twimg.com/media/CT4521TWwAEvMyu.jpg	1	German_shepherd	0.596461	True	malinois	0.138584	True	bloodhound	0.116197	True
3	666044226329800704	https://pbs.twimg.com/media/CT5Dr8HUEAA-lEu.jpg	1	Rhodesian_ridgeback	0.408143	True	redbone	0.360687	True	miniature_pinscher	0.222752	True
4	666049248165822465	https://pbs.twimg.com/media/CT5IQmsXIAAKY4A.jpg	1	miniature_pinscher	0.560311	True	Rottweiler	0.243682	True	Doberman	0.154629	True

3. Gather additional informations directly from twitter¶

From the twitter archive data loaded as df_archive_clean, "retweet count" and "favorite count" ("like") are missing.
We gather this additional information using twitter API. Using the tweet IDs within df_archive_clean we gather all this missing info through queries towards Twitter's API.
The gathering process here is the following:

For each tweet ID in the WeRateDogs archive (df_archive_clean),
We query the Twitter API and get a tweet JSON data
We store the entire set of JSON data in a file called tweet_json.txt, on a new line
Once tweet_json.txt is completed, we read it line by line into a pandas DataFrame (at a minimum with tweet ID, retweet count, favorite count)

In [53]:

# Create twitter API object to gather twitter data
# Get the key, token and secrets
# - keys, token and secrets are hidden here -


# Create the api object
# set the wait_on_rate_limit to True to automatically wait for rate limits to refill
# set wait_on_rate_limit_notify to True to print a notification when Tweepy is waiting for rate limits to refill
api = tweepy.API(auth, wait_on_rate_limit=True, wait_on_rate_limit_notify=True)

The next cell query the twitter API and write the content into "tweet_json.txt" file.
Due to the rate limit from twitter, this code run for more or less 30 minutes !

In [58]:

# Just to keep the tweet IDs with no status found
no_status_tweet_ids = []

# Start code timer
start = timer()

# Create and open a tweet_json.txt file
with open('tweet_json.txt', mode='w') as json_file:
    
    # Iterate over the "tweet_id" column from WeRateDogs twitter archive "df_archive_clean"
    for tweet_id in np.nditer(df_archive_clean.tweet_id):
        
        # Try...except in case some tweets IDs do not lead to any status
        try:
            # Get the object associated with tweet_id
            a_tweet = api.get_status(tweet_id, tweet_mode='extended')
            # Get the tweet JSON info from the object, and write it into a file
            json_file.write(json.dumps(a_tweet._json)+'\n')
        except tweepy.TweepError as e:
            print("Error Tweet_Id: " + str(tweet_id) + " : " + str(e))
            # Keep the Ids with no status found
            no_status_tweet_ids.append(str(tweet_id))
        
        # Print out the tweet ID after it was queried
        print("Successfully queried - Tweet ID: " + str(tweet_id))
        
# End code timer
stop = timer()

Successfully queried - Tweet ID: 892420643555336193
Successfully queried - Tweet ID: 892177421306343426
Successfully queried - Tweet ID: 891815181378084864
Successfully queried - Tweet ID: 891689557279858688
Successfully queried - Tweet ID: 891327558926688256
Successfully queried - Tweet ID: 891087950875897856
Successfully queried - Tweet ID: 890971913173991426
Successfully queried - Tweet ID: 890729181411237888
Successfully queried - Tweet ID: 890609185150312448
Successfully queried - Tweet ID: 890240255349198849
Successfully queried - Tweet ID: 890006608113172480
Successfully queried - Tweet ID: 889880896479866881
Successfully queried - Tweet ID: 889665388333682689
Successfully queried - Tweet ID: 889638837579907072
Successfully queried - Tweet ID: 889531135344209921
Successfully queried - Tweet ID: 889278841981685760
Successfully queried - Tweet ID: 888917238123831296
Successfully queried - Tweet ID: 888804989199671297
Successfully queried - Tweet ID: 888554962724278272
Error Tweet_Id: 888202515573088257 : [{'code': 144, 'message': 'No status found with that ID.'}]
Successfully queried - Tweet ID: 888202515573088257
Successfully queried - Tweet ID: 888078434458587136
Successfully queried - Tweet ID: 887705289381826560
Successfully queried - Tweet ID: 887517139158093824
Successfully queried - Tweet ID: 887473957103951883
Successfully queried - Tweet ID: 887343217045368832
Successfully queried - Tweet ID: 887101392804085760
Successfully queried - Tweet ID: 886983233522544640
Successfully queried - Tweet ID: 886736880519319552
Successfully queried - Tweet ID: 886680336477933568
Successfully queried - Tweet ID: 886366144734445568
Successfully queried - Tweet ID: 886267009285017600
Successfully queried - Tweet ID: 886258384151887873
Successfully queried - Tweet ID: 886054160059072513
Successfully queried - Tweet ID: 885984800019947520
Successfully queried - Tweet ID: 885528943205470208
Successfully queried - Tweet ID: 885518971528720385
Successfully queried - Tweet ID: 885311592912609280
Successfully queried - Tweet ID: 885167619883638784
Successfully queried - Tweet ID: 884925521741709313
Successfully queried - Tweet ID: 884876753390489601
Successfully queried - Tweet ID: 884562892145688576
Successfully queried - Tweet ID: 884441805382717440
Successfully queried - Tweet ID: 884247878851493888
Successfully queried - Tweet ID: 884162670584377345
Successfully queried - Tweet ID: 883838122936631299
Successfully queried - Tweet ID: 883482846933004288
Successfully queried - Tweet ID: 883360690899218434
Successfully queried - Tweet ID: 883117836046086144
Successfully queried - Tweet ID: 882992080364220416
Successfully queried - Tweet ID: 882762694511734784
Successfully queried - Tweet ID: 882627270321602560
Successfully queried - Tweet ID: 882268110199369728
Successfully queried - Tweet ID: 882045870035918850
Successfully queried - Tweet ID: 881906580714921986
Successfully queried - Tweet ID: 881666595344535552
Successfully queried - Tweet ID: 881633300179243008
Successfully queried - Tweet ID: 881536004380872706
Successfully queried - Tweet ID: 881268444196462592
Successfully queried - Tweet ID: 880935762899988482
Successfully queried - Tweet ID: 880872448815771648
Successfully queried - Tweet ID: 880465832366813184
Successfully queried - Tweet ID: 880221127280381952
Successfully queried - Tweet ID: 880095782870896641
Successfully queried - Tweet ID: 879862464715927552
Successfully queried - Tweet ID: 879674319642796034
Successfully queried - Tweet ID: 879492040517615616
Successfully queried - Tweet ID: 879415818425184262
Successfully queried - Tweet ID: 879376492567855104
Successfully queried - Tweet ID: 879130579576475649
Successfully queried - Tweet ID: 879050749262655488
Successfully queried - Tweet ID: 879008229531029506
Successfully queried - Tweet ID: 878776093423087618
Successfully queried - Tweet ID: 878604707211726852
Successfully queried - Tweet ID: 878404777348136964
Successfully queried - Tweet ID: 878316110768087041
Successfully queried - Tweet ID: 878281511006478336
Successfully queried - Tweet ID: 878057613040115712
Successfully queried - Tweet ID: 877736472329191424
Successfully queried - Tweet ID: 877611172832227328
Successfully queried - Tweet ID: 877556246731214848
Successfully queried - Tweet ID: 877316821321428993
Successfully queried - Tweet ID: 877201837425926144
Successfully queried - Tweet ID: 876838120628539392
Successfully queried - Tweet ID: 876537666061221889
Successfully queried - Tweet ID: 876484053909872640
Successfully queried - Tweet ID: 876120275196170240
Successfully queried - Tweet ID: 875747767867523072
Successfully queried - Tweet ID: 875144289856114688
Successfully queried - Tweet ID: 875097192612077568
Successfully queried - Tweet ID: 875021211251597312
Successfully queried - Tweet ID: 874680097055178752
Successfully queried - Tweet ID: 874434818259525634
Successfully queried - Tweet ID: 874296783580663808
Successfully queried - Tweet ID: 874057562936811520
Successfully queried - Tweet ID: 874012996292530176
Error Tweet_Id: 873697596434513921 : [{'code': 144, 'message': 'No status found with that ID.'}]
Successfully queried - Tweet ID: 873697596434513921
Successfully queried - Tweet ID: 873580283840344065
Successfully queried - Tweet ID: 873337748698140672
Successfully queried - Tweet ID: 873213775632977920
Successfully queried - Tweet ID: 872967104147763200
Successfully queried - Tweet ID: 872820683541237760
Error Tweet_Id: 872668790621863937 : [{'code': 144, 'message': 'No status found with that ID.'}]
Successfully queried - Tweet ID: 872668790621863937
Successfully queried - Tweet ID: 872620804844003328
Successfully queried - Tweet ID: 872486979161796608
Error Tweet_Id: 872261713294495745 : [{'code': 144, 'message': 'No status found with that ID.'}]
Successfully queried - Tweet ID: 872261713294495745
Successfully queried - Tweet ID: 872122724285648897
Successfully queried - Tweet ID: 871879754684805121
Successfully queried - Tweet ID: 871762521631449091
Successfully queried - Tweet ID: 871515927908634625
Successfully queried - Tweet ID: 871166179821445120
Successfully queried - Tweet ID: 871102520638267392
Successfully queried - Tweet ID: 871032628920680449
Successfully queried - Tweet ID: 870804317367881728
Successfully queried - Tweet ID: 870726314365509632
Successfully queried - Tweet ID: 870656317836468226
Successfully queried - Tweet ID: 870374049280663552
Successfully queried - Tweet ID: 870308999962521604
Successfully queried - Tweet ID: 870063196459192321
Error Tweet_Id: 869988702071779329 : [{'code': 144, 'message': 'No status found with that ID.'}]
Successfully queried - Tweet ID: 869988702071779329
Successfully queried - Tweet ID: 869772420881756160
Successfully queried - Tweet ID: 869702957897576449
Successfully queried - Tweet ID: 869596645499047938
Successfully queried - Tweet ID: 869227993411051520
Successfully queried - Tweet ID: 868880397819494401
Successfully queried - Tweet ID: 868639477480148993
Successfully queried - Tweet ID: 868622495443632128
Successfully queried - Tweet ID: 868552278524837888
Successfully queried - Tweet ID: 867900495410671616
Successfully queried - Tweet ID: 867774946302451713
Successfully queried - Tweet ID: 867421006826221569
Successfully queried - Tweet ID: 867072653475098625
Successfully queried - Tweet ID: 867051520902168576
Error Tweet_Id: 866816280283807744 : [{'code': 144, 'message': 'No status found with that ID.'}]
Successfully queried - Tweet ID: 866816280283807744
Successfully queried - Tweet ID: 866720684873056260
Successfully queried - Tweet ID: 866686824827068416
Successfully queried - Tweet ID: 866450705531457537
Successfully queried - Tweet ID: 866334964761202691
Successfully queried - Tweet ID: 866094527597207552
Successfully queried - Tweet ID: 865718153858494464
Successfully queried - Tweet ID: 865359393868664832
Successfully queried - Tweet ID: 865006731092295680
Successfully queried - Tweet ID: 864873206498414592
Successfully queried - Tweet ID: 864279568663928832
Successfully queried - Tweet ID: 864197398364647424
Successfully queried - Tweet ID: 863907417377173506
Successfully queried - Tweet ID: 863553081350529029
Successfully queried - Tweet ID: 863471782782697472
Successfully queried - Tweet ID: 863432100342583297
Successfully queried - Tweet ID: 863427515083354112
Successfully queried - Tweet ID: 863079547188785154
Successfully queried - Tweet ID: 863062471531167744
Successfully queried - Tweet ID: 862831371563274240
Successfully queried - Tweet ID: 862722525377298433
Successfully queried - Tweet ID: 862457590147678208
Successfully queried - Tweet ID: 862096992088072192
Error Tweet_Id: 861769973181624320 : [{'code': 144, 'message': 'No status found with that ID.'}]
Successfully queried - Tweet ID: 861769973181624320
Successfully queried - Tweet ID: 861383897657036800
Successfully queried - Tweet ID: 861288531465048066
Successfully queried - Tweet ID: 861005113778896900
Successfully queried - Tweet ID: 860981674716409858
Successfully queried - Tweet ID: 860924035999428608
Successfully queried - Tweet ID: 860563773140209665
Successfully queried - Tweet ID: 860524505164394496
Successfully queried - Tweet ID: 860276583193509888
Successfully queried - Tweet ID: 860184849394610176
Successfully queried - Tweet ID: 860177593139703809
Successfully queried - Tweet ID: 859924526012018688
Successfully queried - Tweet ID: 859851578198683649
Successfully queried - Tweet ID: 859607811541651456
Successfully queried - Tweet ID: 859196978902773760
Successfully queried - Tweet ID: 859074603037188101
Successfully queried - Tweet ID: 858860390427611136
Successfully queried - Tweet ID: 858843525470990336
Successfully queried - Tweet ID: 858471635011153920
Successfully queried - Tweet ID: 858107933456039936
Successfully queried - Tweet ID: 857989990357356544
Successfully queried - Tweet ID: 857746408056729600
Successfully queried - Tweet ID: 857393404942143489
Successfully queried - Tweet ID: 857263160327368704
Successfully queried - Tweet ID: 857214891891077121
Successfully queried - Tweet ID: 857062103051644929
Successfully queried - Tweet ID: 857029823797047296
Error Tweet_Id: 856602993587888130 : [{'code': 144, 'message': 'No status found with that ID.'}]
Successfully queried - Tweet ID: 856602993587888130
Successfully queried - Tweet ID: 856543823941562368
Successfully queried - Tweet ID: 856526610513747968
Successfully queried - Tweet ID: 856330835276025856
Successfully queried - Tweet ID: 856288084350160898
Successfully queried - Tweet ID: 856282028240666624
Successfully queried - Tweet ID: 855862651834028034
Successfully queried - Tweet ID: 855860136149123072
Successfully queried - Tweet ID: 855857698524602368
Successfully queried - Tweet ID: 855851453814013952
Successfully queried - Tweet ID: 855818117272018944
Successfully queried - Tweet ID: 855459453768019968
Successfully queried - Tweet ID: 855245323840757760
Successfully queried - Tweet ID: 855138241867124737
Successfully queried - Tweet ID: 854732716440526848
Successfully queried - Tweet ID: 854482394044301312
Successfully queried - Tweet ID: 854365224396361728
Successfully queried - Tweet ID: 854120357044912130
Successfully queried - Tweet ID: 854010172552949760
Successfully queried - Tweet ID: 853760880890318849
Successfully queried - Tweet ID: 853639147608842240
Successfully queried - Tweet ID: 853299958564483072
Successfully queried - Tweet ID: 852936405516943360
Successfully queried - Tweet ID: 852912242202992640
Successfully queried - Tweet ID: 852672615818899456
Successfully queried - Tweet ID: 852553447878664193
Successfully queried - Tweet ID: 852311364735569921
Successfully queried - Tweet ID: 852226086759018497
Successfully queried - Tweet ID: 852189679701164033
Successfully queried - Tweet ID: 851953902622658560
Successfully queried - Tweet ID: 851861385021730816
Successfully queried - Tweet ID: 851591660324737024
Successfully queried - Tweet ID: 851464819735769094
Successfully queried - Tweet ID: 851224888060895234
Successfully queried - Tweet ID: 850753642995093505
Successfully queried - Tweet ID: 850380195714523136
Successfully queried - Tweet ID: 850333567704068097
Successfully queried - Tweet ID: 850145622816686080
Successfully queried - Tweet ID: 850019790995546112
Successfully queried - Tweet ID: 849776966551130114
Successfully queried - Tweet ID: 849668094696017920
Successfully queried - Tweet ID: 849412302885593088
Successfully queried - Tweet ID: 849336543269576704
Successfully queried - Tweet ID: 849051919805034497
Successfully queried - Tweet ID: 848690551926992896
Successfully queried - Tweet ID: 848324959059550208
Successfully queried - Tweet ID: 848213670039564288
Successfully queried - Tweet ID: 848212111729840128
Successfully queried - Tweet ID: 847978865427394560
Successfully queried - Tweet ID: 847971574464610304
Successfully queried - Tweet ID: 847962785489326080
Successfully queried - Tweet ID: 847842811428974592
Successfully queried - Tweet ID: 847617282490613760
Successfully queried - Tweet ID: 847606175596138505
Successfully queried - Tweet ID: 847251039262605312
Successfully queried - Tweet ID: 847157206088847362
Successfully queried - Tweet ID: 847116187444137987
Successfully queried - Tweet ID: 846874817362120707
Successfully queried - Tweet ID: 846514051647705089
Successfully queried - Tweet ID: 846505985330044928
Successfully queried - Tweet ID: 846153765933735936
Successfully queried - Tweet ID: 846139713627017216
Successfully queried - Tweet ID: 846042936437604353
Successfully queried - Tweet ID: 845812042753855489
Successfully queried - Tweet ID: 845677943972139009
Error Tweet_Id: 845459076796616705 : [{'code': 144, 'message': 'No status found with that ID.'}]
Successfully queried - Tweet ID: 845459076796616705
Successfully queried - Tweet ID: 845397057150107648
Successfully queried - Tweet ID: 845306882940190720
Successfully queried - Tweet ID: 845098359547420673
Successfully queried - Tweet ID: 844979544864018432
Successfully queried - Tweet ID: 844973813909606400
Error Tweet_Id: 844704788403113984 : [{'code': 144, 'message': 'No status found with that ID.'}]
Successfully queried - Tweet ID: 844704788403113984
Successfully queried - Tweet ID: 844580511645339650
Successfully queried - Tweet ID: 844223788422217728
Successfully queried - Tweet ID: 843981021012017153
Successfully queried - Tweet ID: 843856843873095681
Successfully queried - Tweet ID: 843604394117681152
Successfully queried - Tweet ID: 843235543001513987
Error Tweet_Id: 842892208864923648 : [{'code': 144, 'message': 'No status found with that ID.'}]
Successfully queried - Tweet ID: 842892208864923648
Successfully queried - Tweet ID: 842846295480000512
Successfully queried - Tweet ID: 842765311967449089
Successfully queried - Tweet ID: 842535590457499648
Successfully queried - Tweet ID: 842163532590374912
Successfully queried - Tweet ID: 842115215311396866
Successfully queried - Tweet ID: 841833993020538882
Successfully queried - Tweet ID: 841680585030541313
Successfully queried - Tweet ID: 841439858740625411
Successfully queried - Tweet ID: 841320156043304961
Successfully queried - Tweet ID: 841314665196081154
Successfully queried - Tweet ID: 841077006473256960
Successfully queried - Tweet ID: 840761248237133825
Successfully queried - Tweet ID: 840728873075638272
Successfully queried - Tweet ID: 840698636975636481
Successfully queried - Tweet ID: 840696689258311684
Successfully queried - Tweet ID: 840632337062862849
Successfully queried - Tweet ID: 840370681858686976
Successfully queried - Tweet ID: 840268004936019968
Successfully queried - Tweet ID: 839990271299457024
Successfully queried - Tweet ID: 839549326359670784
Successfully queried - Tweet ID: 839290600511926273
Successfully queried - Tweet ID: 839239871831150596
Successfully queried - Tweet ID: 838952994649550848
Successfully queried - Tweet ID: 838921590096166913
Successfully queried - Tweet ID: 838916489579200512
Successfully queried - Tweet ID: 838831947270979586
Successfully queried - Tweet ID: 838561493054533637
Successfully queried - Tweet ID: 838476387338051585
Successfully queried - Tweet ID: 838201503651401729
Successfully queried - Tweet ID: 838150277551247360
Successfully queried - Tweet ID: 838085839343206401
Successfully queried - Tweet ID: 838083903487373313
Successfully queried - Tweet ID: 837820167694528512
Successfully queried - Tweet ID: 837482249356513284
Successfully queried - Tweet ID: 837471256429613056
Successfully queried - Tweet ID: 837366284874571778
Successfully queried - Tweet ID: 837110210464448512
Error Tweet_Id: 837012587749474308 : [{'code': 144, 'message': 'No status found with that ID.'}]
Successfully queried - Tweet ID: 837012587749474308
Successfully queried - Tweet ID: 836989968035819520
Successfully queried - Tweet ID: 836753516572119041
Successfully queried - Tweet ID: 836677758902222849
Successfully queried - Tweet ID: 836648853927522308
Successfully queried - Tweet ID: 836397794269200385
Successfully queried - Tweet ID: 836380477523124226
Successfully queried - Tweet ID: 836260088725786625
Successfully queried - Tweet ID: 836001077879255040
Successfully queried - Tweet ID: 835685285446955009
Successfully queried - Tweet ID: 835574547218894849
Successfully queried - Tweet ID: 835536468978302976
Successfully queried - Tweet ID: 835309094223372289
Successfully queried - Tweet ID: 835297930240217089
Successfully queried - Tweet ID: 835264098648616962
Successfully queried - Tweet ID: 835246439529840640
Successfully queried - Tweet ID: 835172783151792128
Successfully queried - Tweet ID: 835152434251116546
Successfully queried - Tweet ID: 834931633769889797
Successfully queried - Tweet ID: 834786237630337024
Successfully queried - Tweet ID: 834574053763584002
Successfully queried - Tweet ID: 834477809192075265
Successfully queried - Tweet ID: 834458053273591808
Successfully queried - Tweet ID: 834209720923721728
Successfully queried - Tweet ID: 834167344700198914
Successfully queried - Tweet ID: 834089966724603904
Successfully queried - Tweet ID: 834086379323871233
Successfully queried - Tweet ID: 833863086058651648
Successfully queried - Tweet ID: 833826103416520705
Successfully queried - Tweet ID: 833732339549220864
Successfully queried - Tweet ID: 833722901757046785
Successfully queried - Tweet ID: 833479644947025920
Successfully queried - Tweet ID: 833124694597443584
Successfully queried - Tweet ID: 832998151111966721
Successfully queried - Tweet ID: 832769181346996225
Successfully queried - Tweet ID: 832757312314028032
Successfully queried - Tweet ID: 832682457690300417
Successfully queried - Tweet ID: 832645525019123713
Successfully queried - Tweet ID: 832636094638288896
Successfully queried - Tweet ID: 832397543355072512
Successfully queried - Tweet ID: 832369877331693569
Successfully queried - Tweet ID: 832273440279240704
Successfully queried - Tweet ID: 832215909146226688
Successfully queried - Tweet ID: 832215726631055365
Successfully queried - Tweet ID: 832088576586297345
Successfully queried - Tweet ID: 832040443403784192
Successfully queried - Tweet ID: 832032802820481025
Successfully queried - Tweet ID: 831939777352105988
Successfully queried - Tweet ID: 831926988323639298
Successfully queried - Tweet ID: 831911600680497154
Successfully queried - Tweet ID: 831670449226514432
Successfully queried - Tweet ID: 831650051525054464
Successfully queried - Tweet ID: 831552930092285952
Successfully queried - Tweet ID: 831322785565769729
Successfully queried - Tweet ID: 831315979191906304
Successfully queried - Tweet ID: 831309418084069378
Successfully queried - Tweet ID: 831262627380748289
Successfully queried - Tweet ID: 830956169170665475
Successfully queried - Tweet ID: 830583320585068544
Successfully queried - Tweet ID: 830173239259324417
Successfully queried - Tweet ID: 830097400375152640
Successfully queried - Tweet ID: 829878982036299777
Successfully queried - Tweet ID: 829861396166877184
Successfully queried - Tweet ID: 829501995190984704
Successfully queried - Tweet ID: 829449946868879360
Successfully queried - Tweet ID: 829374341691346946
Successfully queried - Tweet ID: 829141528400556032
Successfully queried - Tweet ID: 829011960981237760
Successfully queried - Tweet ID: 828801551087042563
Successfully queried - Tweet ID: 828770345708580865
Successfully queried - Tweet ID: 828708714936930305
Successfully queried - Tweet ID: 828650029636317184
Successfully queried - Tweet ID: 828409743546925057
Successfully queried - Tweet ID: 828408677031882754
Successfully queried - Tweet ID: 828381636999917570
Successfully queried - Tweet ID: 828376505180889089
Successfully queried - Tweet ID: 828372645993398273
Successfully queried - Tweet ID: 828361771580813312
Successfully queried - Tweet ID: 828046555563323392
Successfully queried - Tweet ID: 828011680017821696
Successfully queried - Tweet ID: 827933404142436356
Successfully queried - Tweet ID: 827653905312006145
Successfully queried - Tweet ID: 827600520311402496
Successfully queried - Tweet ID: 827324948884643840
Error Tweet_Id: 827228250799742977 : [{'code': 144, 'message': 'No status found with that ID.'}]
Successfully queried - Tweet ID: 827228250799742977
Successfully queried - Tweet ID: 827199976799354881
Successfully queried - Tweet ID: 826958653328592898
Successfully queried - Tweet ID: 826848821049180160
Successfully queried - Tweet ID: 826615380357632002
Successfully queried - Tweet ID: 826598799820865537
Successfully queried - Tweet ID: 826598365270007810
Successfully queried - Tweet ID: 826476773533745153
Successfully queried - Tweet ID: 826240494070030336
Successfully queried - Tweet ID: 826204788643753985
Successfully queried - Tweet ID: 826115272272650244
Successfully queried - Tweet ID: 825876512159186944
Successfully queried - Tweet ID: 825829644528148480
Successfully queried - Tweet ID: 825535076884762624
Successfully queried - Tweet ID: 825147591692263424
Successfully queried - Tweet ID: 825120256414846976
Successfully queried - Tweet ID: 825026590719483904
Successfully queried - Tweet ID: 824796380199809024
Successfully queried - Tweet ID: 824775126675836928
Successfully queried - Tweet ID: 824663926340194305
Successfully queried - Tweet ID: 824325613288833024
Successfully queried - Tweet ID: 824297048279236611
Successfully queried - Tweet ID: 824025158776213504
Successfully queried - Tweet ID: 823939628516474880
Successfully queried - Tweet ID: 823719002937630720
Successfully queried - Tweet ID: 823699002998870016
Successfully queried - Tweet ID: 823581115634085888
Successfully queried - Tweet ID: 823333489516937216
Successfully queried - Tweet ID: 823322678127919110
Successfully queried - Tweet ID: 823269594223824897
Successfully queried - Tweet ID: 822975315408461824
Successfully queried - Tweet ID: 822872901745569793
Successfully queried - Tweet ID: 822859134160621569
Successfully queried - Tweet ID: 822647212903690241
Successfully queried - Tweet ID: 822610361945911296
Successfully queried - Tweet ID: 822489057087389700
Successfully queried - Tweet ID: 822462944365645825
Successfully queried - Tweet ID: 822244816520155136
Successfully queried - Tweet ID: 822163064745328640
Successfully queried - Tweet ID: 821886076407029760
Successfully queried - Tweet ID: 821813639212650496
Successfully queried - Tweet ID: 821765923262631936
Successfully queried - Tweet ID: 821522889702862852
Successfully queried - Tweet ID: 821421320206483457
Successfully queried - Tweet ID: 821407182352777218
Successfully queried - Tweet ID: 821153421864615936
Successfully queried - Tweet ID: 821149554670182400
Successfully queried - Tweet ID: 821107785811234820
Successfully queried - Tweet ID: 821044531881721856
Successfully queried - Tweet ID: 820837357901512704
Successfully queried - Tweet ID: 820749716845686786
Successfully queried - Tweet ID: 820690176645140481
Successfully queried - Tweet ID: 820494788566847489
Successfully queried - Tweet ID: 820446719150292993
Successfully queried - Tweet ID: 820314633777061888
Successfully queried - Tweet ID: 820078625395449857
Successfully queried - Tweet ID: 820013781606658049
Successfully queried - Tweet ID: 819952236453363712
Successfully queried - Tweet ID: 819924195358416896
Successfully queried - Tweet ID: 819711362133872643
Successfully queried - Tweet ID: 819588359383371776
Successfully queried - Tweet ID: 819347104292290561
Successfully queried - Tweet ID: 819238181065359361
Successfully queried - Tweet ID: 819227688460238848
Successfully queried - Tweet ID: 819015337530290176
Successfully queried - Tweet ID: 819015331746349057
Successfully queried - Tweet ID: 819006400881917954
Successfully queried - Tweet ID: 819004803107983360
Successfully queried - Tweet ID: 818646164899774465
Successfully queried - Tweet ID: 818627210458333184
Successfully queried - Tweet ID: 818614493328580609
Successfully queried - Tweet ID: 818588835076603904
Successfully queried - Tweet ID: 818536468981415936
Successfully queried - Tweet ID: 818307523543449600
Successfully queried - Tweet ID: 818259473185828864
Successfully queried - Tweet ID: 818145370475810820
Successfully queried - Tweet ID: 817908911860748288
Successfully queried - Tweet ID: 817827839487737858
Successfully queried - Tweet ID: 817777686764523521
Successfully queried - Tweet ID: 817536400337801217
Successfully queried - Tweet ID: 817502432452313088
Successfully queried - Tweet ID: 817423860136083457
Successfully queried - Tweet ID: 817415592588222464
Successfully queried - Tweet ID: 817181837579653120
Successfully queried - Tweet ID: 817171292965273600
Successfully queried - Tweet ID: 817120970343411712
Successfully queried - Tweet ID: 817056546584727552
Successfully queried - Tweet ID: 816829038950027264
Successfully queried - Tweet ID: 816816676327063552
Successfully queried - Tweet ID: 816697700272001025
Successfully queried - Tweet ID: 816450570814898180
Successfully queried - Tweet ID: 816336735214911488
Successfully queried - Tweet ID: 816091915477250048
Successfully queried - Tweet ID: 816062466425819140
Successfully queried - Tweet ID: 816014286006976512
Successfully queried - Tweet ID: 815990720817401858
Successfully queried - Tweet ID: 815966073409433600
Successfully queried - Tweet ID: 815745968457060357
Successfully queried - Tweet ID: 815736392542261248
Successfully queried - Tweet ID: 815639385530101762
Successfully queried - Tweet ID: 815390420867969024
Successfully queried - Tweet ID: 814986499976527872
Successfully queried - Tweet ID: 814638523311648768
Successfully queried - Tweet ID: 814578408554463233
Successfully queried - Tweet ID: 814530161257443328
Successfully queried - Tweet ID: 814153002265309185
Successfully queried - Tweet ID: 813944609378369540
Successfully queried - Tweet ID: 813910438903693312
Successfully queried - Tweet ID: 813812741911748608
Successfully queried - Tweet ID: 813800681631023104
Successfully queried - Tweet ID: 813217897535406080
Successfully queried - Tweet ID: 813202720496779264
Successfully queried - Tweet ID: 813187593374461952
Successfully queried - Tweet ID: 813172488309972993
Successfully queried - Tweet ID: 813157409116065792
Successfully queried - Tweet ID: 813142292504645637
Successfully queried - Tweet ID: 813130366689148928
Successfully queried - Tweet ID: 813127251579564032
Successfully queried - Tweet ID: 813112105746448384
Successfully queried - Tweet ID: 813096984823349248
Successfully queried - Tweet ID: 813081950185472002
Successfully queried - Tweet ID: 813066809284972545
Successfully queried - Tweet ID: 813051746834595840
Successfully queried - Tweet ID: 812781120811126785
Error Tweet_Id: 812747805718642688 : [{'code': 144, 'message': 'No status found with that ID.'}]
Successfully queried - Tweet ID: 812747805718642688
Successfully queried - Tweet ID: 812709060537683968
Successfully queried - Tweet ID: 812503143955202048
Successfully queried - Tweet ID: 812466873996607488
Successfully queried - Tweet ID: 812372279581671427
Successfully queried - Tweet ID: 811985624773361665
Successfully queried - Tweet ID: 811744202451197953
Successfully queried - Tweet ID: 811647686436880384
Successfully queried - Tweet ID: 811627233043480576
Successfully queried - Tweet ID: 811386762094317568
Successfully queried - Tweet ID: 810984652412424192
Successfully queried - Tweet ID: 810896069567610880
Successfully queried - Tweet ID: 810657578271330305
Successfully queried - Tweet ID: 810284430598270976
Successfully queried - Tweet ID: 810254108431155201
Successfully queried - Tweet ID: 809920764300447744
Successfully queried - Tweet ID: 809808892968534016
Successfully queried - Tweet ID: 809448704142938112
Successfully queried - Tweet ID: 809220051211603969
Successfully queried - Tweet ID: 809084759137812480
Successfully queried - Tweet ID: 808838249661788160
Successfully queried - Tweet ID: 808733504066486276
Successfully queried - Tweet ID: 808501579447930884
Successfully queried - Tweet ID: 808344865868283904
Successfully queried - Tweet ID: 808134635716833280
Successfully queried - Tweet ID: 808106460588765185
Successfully queried - Tweet ID: 808001312164028416
Successfully queried - Tweet ID: 807621403335917568
Successfully queried - Tweet ID: 807106840509214720
Successfully queried - Tweet ID: 807059379405148160
Successfully queried - Tweet ID: 807010152071229440
Successfully queried - Tweet ID: 806629075125202948
Successfully queried - Tweet ID: 806620845233815552
Successfully queried - Tweet ID: 806576416489959424
Successfully queried - Tweet ID: 806542213899489280
Successfully queried - Tweet ID: 806242860592926720
Successfully queried - Tweet ID: 806219024703037440
Successfully queried - Tweet ID: 805958939288408065
Successfully queried - Tweet ID: 805932879469572096
Successfully queried - Tweet ID: 805826884734976000
Successfully queried - Tweet ID: 805823200554876929
Successfully queried - Tweet ID: 805520635690676224
Successfully queried - Tweet ID: 805487436403003392
Successfully queried - Tweet ID: 805207613751304193
Successfully queried - Tweet ID: 804738756058218496
Successfully queried - Tweet ID: 804475857670639616
Successfully queried - Tweet ID: 804413760345620481
Successfully queried - Tweet ID: 804026241225523202
Successfully queried - Tweet ID: 803773340896923648
Successfully queried - Tweet ID: 803692223237865472
Successfully queried - Tweet ID: 803638050916102144
Successfully queried - Tweet ID: 803380650405482500
Successfully queried - Tweet ID: 803321560782307329
Successfully queried - Tweet ID: 803276597545603072
Successfully queried - Tweet ID: 802952499103731712
Successfully queried - Tweet ID: 802624713319034886
Successfully queried - Tweet ID: 802600418706604034
Successfully queried - Tweet ID: 802572683846291456
Successfully queried - Tweet ID: 802323869084381190
Successfully queried - Tweet ID: 802265048156610565
Error Tweet_Id: 802247111496568832 : [{'code': 144, 'message': 'No status found with that ID.'}]
Successfully queried - Tweet ID: 802247111496568832
Successfully queried - Tweet ID: 802239329049477120
Successfully queried - Tweet ID: 802185808107208704
Successfully queried - Tweet ID: 801958328846974976
Successfully queried - Tweet ID: 801854953262350336
Successfully queried - Tweet ID: 801538201127157760
Successfully queried - Tweet ID: 801285448605831168
Successfully queried - Tweet ID: 801167903437357056
Successfully queried - Tweet ID: 801127390143516673
Successfully queried - Tweet ID: 801115127852503040
Successfully queried - Tweet ID: 800859414831898624
Successfully queried - Tweet ID: 800855607700029440
Successfully queried - Tweet ID: 800751577355128832
Successfully queried - Tweet ID: 800513324630806528
Successfully queried - Tweet ID: 800459316964663297
Successfully queried - Tweet ID: 800443802682937345
Successfully queried - Tweet ID: 800388270626521089
Successfully queried - Tweet ID: 800188575492947969
Successfully queried - Tweet ID: 800141422401830912
Successfully queried - Tweet ID: 800018252395122689
Successfully queried - Tweet ID: 799774291445383169
Successfully queried - Tweet ID: 799757965289017345
Successfully queried - Tweet ID: 799422933579902976
Successfully queried - Tweet ID: 799308762079035393
Successfully queried - Tweet ID: 799297110730567681
Successfully queried - Tweet ID: 799063482566066176
Successfully queried - Tweet ID: 798933969379225600
Successfully queried - Tweet ID: 798925684722855936
Successfully queried - Tweet ID: 798705661114773508
Successfully queried - Tweet ID: 798701998996647937
Successfully queried - Tweet ID: 798697898615730177
Successfully queried - Tweet ID: 798694562394996736
Successfully queried - Tweet ID: 798686750113755136
Successfully queried - Tweet ID: 798682547630837760
Successfully queried - Tweet ID: 798673117451325440
Successfully queried - Tweet ID: 798665375516884993
Successfully queried - Tweet ID: 798644042770751489
Successfully queried - Tweet ID: 798628517273620480
Successfully queried - Tweet ID: 798585098161549313
Successfully queried - Tweet ID: 798576900688019456
Successfully queried - Tweet ID: 798340744599797760
Successfully queried - Tweet ID: 798209839306514432
Successfully queried - Tweet ID: 797971864723324932
Successfully queried - Tweet ID: 797545162159308800
Successfully queried - Tweet ID: 797236660651966464
Successfully queried - Tweet ID: 797165961484890113
Successfully queried - Tweet ID: 796904159865868288
Successfully queried - Tweet ID: 796865951799083009
Successfully queried - Tweet ID: 796759840936919040
Successfully queried - Tweet ID: 796563435802726400
Successfully queried - Tweet ID: 796484825502875648
Successfully queried - Tweet ID: 796387464403357696
Successfully queried - Tweet ID: 796177847564038144
Successfully queried - Tweet ID: 796149749086875649
Successfully queried - Tweet ID: 796125600683540480
Successfully queried - Tweet ID: 796116448414461957
Successfully queried - Tweet ID: 796080075804475393
Successfully queried - Tweet ID: 796031486298386433
Successfully queried - Tweet ID: 795464331001561088
Successfully queried - Tweet ID: 795400264262053889
Successfully queried - Tweet ID: 795076730285391872
Successfully queried - Tweet ID: 794983741416415232
Successfully queried - Tweet ID: 794926597468000259
Successfully queried - Tweet ID: 794355576146903043
Successfully queried - Tweet ID: 794332329137291264
Successfully queried - Tweet ID: 794205286408003585
Successfully queried - Tweet ID: 793962221541933056
Successfully queried - Tweet ID: 793845145112371200
Successfully queried - Tweet ID: 793614319594401792
Successfully queried - Tweet ID: 793601777308463104
Successfully queried - Tweet ID: 793500921481273345
Successfully queried - Tweet ID: 793286476301799424
Successfully queried - Tweet ID: 793271401113350145
Successfully queried - Tweet ID: 793256262322548741
Successfully queried - Tweet ID: 793241302385262592
Successfully queried - Tweet ID: 793226087023144960
Successfully queried - Tweet ID: 793210959003287553
Successfully queried - Tweet ID: 793195938047070209
Successfully queried - Tweet ID: 793180763617361921
Successfully queried - Tweet ID: 793165685325201412
Successfully queried - Tweet ID: 793150605191548928
Successfully queried - Tweet ID: 793135492858580992
Successfully queried - Tweet ID: 793120401413079041
Successfully queried - Tweet ID: 792913359805018113
Successfully queried - Tweet ID: 792883833364439040
Successfully queried - Tweet ID: 792773781206999040
Successfully queried - Tweet ID: 792394556390137856
Successfully queried - Tweet ID: 792050063153438720
Successfully queried - Tweet ID: 791821351946420224
Successfully queried - Tweet ID: 791784077045166082
Successfully queried - Tweet ID: 791780927877898241
Successfully queried - Tweet ID: 791774931465953280
Successfully queried - Tweet ID: 791672322847637504
Successfully queried - Tweet ID: 791406955684368384
Successfully queried - Tweet ID: 791312159183634433
Successfully queried - Tweet ID: 791026214425268224
Successfully queried - Tweet ID: 790987426131050500
Successfully queried - Tweet ID: 790946055508652032
Successfully queried - Tweet ID: 790723298204217344
Successfully queried - Tweet ID: 790698755171364864
Successfully queried - Tweet ID: 790581949425475584
Successfully queried - Tweet ID: 790337589677002753
Successfully queried - Tweet ID: 790277117346975746
Successfully queried - Tweet ID: 790227638568808452
Successfully queried - Tweet ID: 789986466051088384
Successfully queried - Tweet ID: 789960241177853952
Successfully queried - Tweet ID: 789903600034189313
Successfully queried - Tweet ID: 789628658055020548
Successfully queried - Tweet ID: 789599242079838210
Successfully queried - Tweet ID: 789530877013393408
Successfully queried - Tweet ID: 789314372632018944
Successfully queried - Tweet ID: 789280767834746880
Successfully queried - Tweet ID: 789268448748703744
Successfully queried - Tweet ID: 789137962068021249
Successfully queried - Tweet ID: 788908386943430656
Successfully queried - Tweet ID: 788765914992902144
Successfully queried - Tweet ID: 788552643979468800
Successfully queried - Tweet ID: 788412144018661376
Successfully queried - Tweet ID: 788178268662984705
Successfully queried - Tweet ID: 788150585577050112
Successfully queried - Tweet ID: 788070120937619456
Successfully queried - Tweet ID: 788039637453406209
Successfully queried - Tweet ID: 787810552592695296
Successfully queried - Tweet ID: 787717603741622272
Successfully queried - Tweet ID: 787397959788929025
Successfully queried - Tweet ID: 787322443945877504
Successfully queried - Tweet ID: 787111942498508800
Successfully queried - Tweet ID: 786963064373534720
Successfully queried - Tweet ID: 786729988674449408
Successfully queried - Tweet ID: 786709082849828864
Successfully queried - Tweet ID: 786664955043049472
Successfully queried - Tweet ID: 786595970293370880
Successfully queried - Tweet ID: 786363235746385920
Successfully queried - Tweet ID: 786286427768250368
Successfully queried - Tweet ID: 786233965241827333
Successfully queried - Tweet ID: 786051337297522688
Successfully queried - Tweet ID: 786036967502913536
Successfully queried - Tweet ID: 785927819176054784
Successfully queried - Tweet ID: 785872687017132033
Successfully queried - Tweet ID: 785639753186217984
Successfully queried - Tweet ID: 785533386513321988
Successfully queried - Tweet ID: 785515384317313025
Successfully queried - Tweet ID: 785264754247995392
Successfully queried - Tweet ID: 785170936622350336
Successfully queried - Tweet ID: 784826020293709826
Successfully queried - Tweet ID: 784517518371221505
Successfully queried - Tweet ID: 784431430411685888
Successfully queried - Tweet ID: 784183165795655680
Successfully queried - Tweet ID: 784057939640352768
Successfully queried - Tweet ID: 783839966405230592
Successfully queried - Tweet ID: 783821107061198850
Successfully queried - Tweet ID: 783695101801398276
Successfully queried - Tweet ID: 783466772167098368
Successfully queried - Tweet ID: 783391753726550016
Successfully queried - Tweet ID: 783347506784731136
Successfully queried - Tweet ID: 783334639985389568
Successfully queried - Tweet ID: 783085703974514689
Successfully queried - Tweet ID: 782969140009107456
Successfully queried - Tweet ID: 782747134529531904
Successfully queried - Tweet ID: 782722598790725632
Successfully queried - Tweet ID: 782598640137187329
Successfully queried - Tweet ID: 782305867769217024
Successfully queried - Tweet ID: 782021823840026624
Successfully queried - Tweet ID: 781955203444699136
Successfully queried - Tweet ID: 781661882474196992
Successfully queried - Tweet ID: 781655249211752448
Successfully queried - Tweet ID: 781524693396357120
Successfully queried - Tweet ID: 781308096455073793
Successfully queried - Tweet ID: 781251288990355457
Successfully queried - Tweet ID: 781163403222056960
Successfully queried - Tweet ID: 780931614150983680
Successfully queried - Tweet ID: 780858289093574656
Successfully queried - Tweet ID: 780800785462489090
Successfully queried - Tweet ID: 780601303617732608
Successfully queried - Tweet ID: 780543529827336192
Successfully queried - Tweet ID: 780496263422808064
Successfully queried - Tweet ID: 780476555013349377
Successfully queried - Tweet ID: 780459368902959104
Successfully queried - Tweet ID: 780192070812196864
Successfully queried - Tweet ID: 780092040432480260
Successfully queried - Tweet ID: 780074436359819264
Successfully queried - Tweet ID: 779834332596887552
Successfully queried - Tweet ID: 779377524342161408
Successfully queried - Tweet ID: 779124354206535695
Successfully queried - Tweet ID: 779123168116150273
Successfully queried - Tweet ID: 779056095788752897
Successfully queried - Tweet ID: 778990705243029504
Successfully queried - Tweet ID: 778774459159379968
Successfully queried - Tweet ID: 778764940568104960
Successfully queried - Tweet ID: 778748913645780993
Successfully queried - Tweet ID: 778650543019483137
Successfully queried - Tweet ID: 778624900596654080
Successfully queried - Tweet ID: 778408200802557953
Successfully queried - Tweet ID: 778396591732486144
Successfully queried - Tweet ID: 778383385161035776
Successfully queried - Tweet ID: 778286810187399168
Successfully queried - Tweet ID: 778039087836069888
Successfully queried - Tweet ID: 778027034220126208
Successfully queried - Tweet ID: 777953400541634568
Successfully queried - Tweet ID: 777885040357281792
Successfully queried - Tweet ID: 777684233540206592
Successfully queried - Tweet ID: 777641927919427584
Successfully queried - Tweet ID: 777621514455814149
Successfully queried - Tweet ID: 777189768882946048
Successfully queried - Tweet ID: 776819012571455488
Successfully queried - Tweet ID: 776813020089548800
Successfully queried - Tweet ID: 776477788987613185
Successfully queried - Tweet ID: 776249906839351296
Successfully queried - Tweet ID: 776218204058357768
Successfully queried - Tweet ID: 776201521193218049
Successfully queried - Tweet ID: 776113305656188928
Successfully queried - Tweet ID: 776088319444877312
Successfully queried - Tweet ID: 775898661951791106
Successfully queried - Tweet ID: 775842724423557120
Successfully queried - Tweet ID: 775733305207554048
Successfully queried - Tweet ID: 775729183532220416
Successfully queried - Tweet ID: 775364825476165632
Successfully queried - Tweet ID: 775350846108426240
Error Tweet_Id: 775096608509886464 : [{'code': 144, 'message': 'No status found with that ID.'}]
Successfully queried - Tweet ID: 775096608509886464
Successfully queried - Tweet ID: 775085132600442880
Successfully queried - Tweet ID: 774757898236878852
Successfully queried - Tweet ID: 774639387460112384
Successfully queried - Tweet ID: 774314403806253056
Successfully queried - Tweet ID: 773985732834758656
Successfully queried - Tweet ID: 773922284943896577
Successfully queried - Tweet ID: 773704687002451968
Successfully queried - Tweet ID: 773670353721753600
Successfully queried - Tweet ID: 773547596996571136
Successfully queried - Tweet ID: 773336787167145985
Successfully queried - Tweet ID: 773308824254029826
Successfully queried - Tweet ID: 773247561583001600
Successfully queried - Tweet ID: 773191612633579521
Successfully queried - Tweet ID: 772877495989305348
Successfully queried - Tweet ID: 772826264096874500
Successfully queried - Tweet ID: 772615324260794368
Successfully queried - Tweet ID: 772581559778025472
Successfully queried - Tweet ID: 772193107915964416
Successfully queried - Tweet ID: 772152991789019136
Successfully queried - Tweet ID: 772117678702071809
Successfully queried - Tweet ID: 772114945936949249
Successfully queried - Tweet ID: 772102971039580160
Successfully queried - Tweet ID: 771908950375665664
Successfully queried - Tweet ID: 771770456517009408
Successfully queried - Tweet ID: 771500966810099713
Successfully queried - Tweet ID: 771380798096281600
Successfully queried - Tweet ID: 771171053431250945
Successfully queried - Tweet ID: 771136648247640064
Successfully queried - Tweet ID: 771102124360998913
Successfully queried - Tweet ID: 771014301343748096
Successfully queried - Tweet ID: 771004394259247104
Successfully queried - Tweet ID: 770787852854652928
Successfully queried - Tweet ID: 770772759874076672
Error Tweet_Id: 770743923962707968 : [{'code': 144, 'message': 'No status found with that ID.'}]
Successfully queried - Tweet ID: 770743923962707968
Successfully queried - Tweet ID: 770655142660169732
Successfully queried - Tweet ID: 770414278348247044
Successfully queried - Tweet ID: 770293558247038976
Successfully queried - Tweet ID: 770093767776997377
Successfully queried - Tweet ID: 770069151037685760
Successfully queried - Tweet ID: 769940425801170949
Successfully queried - Tweet ID: 769695466921623552
Successfully queried - Tweet ID: 769335591808995329
Successfully queried - Tweet ID: 769212283578875904
Successfully queried - Tweet ID: 768970937022709760
Successfully queried - Tweet ID: 768909767477751808
Successfully queried - Tweet ID: 768855141948723200
Successfully queried - Tweet ID: 768609597686943744
Successfully queried - Tweet ID: 768596291618299904
Successfully queried - Tweet ID: 768554158521745409
Successfully queried - Tweet ID: 768473857036525572
Successfully queried - Tweet ID: 768193404517830656
Successfully queried - Tweet ID: 767884188863397888
Successfully queried - Tweet ID: 767754930266464257
Successfully queried - Tweet ID: 767500508068192258
Successfully queried - Tweet ID: 767191397493538821
Successfully queried - Tweet ID: 767122157629476866
Successfully queried - Tweet ID: 766864461642756096
Successfully queried - Tweet ID: 766793450729734144
Successfully queried - Tweet ID: 766714921925144576
Successfully queried - Tweet ID: 766693177336135680
Successfully queried - Tweet ID: 766423258543644672
Successfully queried - Tweet ID: 766313316352462849
Successfully queried - Tweet ID: 766078092750233600
Successfully queried - Tweet ID: 766069199026450432
Successfully queried - Tweet ID: 766008592277377025
Successfully queried - Tweet ID: 765719909049503744
Successfully queried - Tweet ID: 765669560888528897
Successfully queried - Tweet ID: 765395769549590528
Successfully queried - Tweet ID: 765371061932261376
Successfully queried - Tweet ID: 765222098633691136
Successfully queried - Tweet ID: 764857477905154048
Successfully queried - Tweet ID: 764259802650378240
Successfully queried - Tweet ID: 763956972077010945
Successfully queried - Tweet ID: 763837565564780549
Successfully queried - Tweet ID: 763183847194451968
Successfully queried - Tweet ID: 763167063695355904
Successfully queried - Tweet ID: 763103485927849985
Successfully queried - Tweet ID: 762699858130116608
Successfully queried - Tweet ID: 762471784394268675
Successfully queried - Tweet ID: 762464539388485633
Successfully queried - Tweet ID: 762316489655476224
Successfully queried - Tweet ID: 762035686371364864
Successfully queried - Tweet ID: 761976711479193600
Successfully queried - Tweet ID: 761750502866649088
Successfully queried - Tweet ID: 761745352076779520
Successfully queried - Tweet ID: 761672994376806400
Successfully queried - Tweet ID: 761599872357261312
Successfully queried - Tweet ID: 761371037149827077
Successfully queried - Tweet ID: 761334018830917632
Successfully queried - Tweet ID: 761292947749015552
Successfully queried - Tweet ID: 761227390836215808
Successfully queried - Tweet ID: 761004547850530816
Successfully queried - Tweet ID: 760893934457552897
Successfully queried - Tweet ID: 760656994973933572
Successfully queried - Tweet ID: 760641137271070720
Successfully queried - Tweet ID: 760539183865880579
Successfully queried - Tweet ID: 760521673607086080
Successfully queried - Tweet ID: 760290219849637889
Successfully queried - Tweet ID: 760252756032651264
Successfully queried - Tweet ID: 760190180481531904
Successfully queried - Tweet ID: 760153949710192640
Successfully queried - Tweet ID: 759943073749200896
Successfully queried - Tweet ID: 759923798737051648
Successfully queried - Tweet ID: 759846353224826880
Successfully queried - Tweet ID: 759793422261743616
Successfully queried - Tweet ID: 759566828574212096
Successfully queried - Tweet ID: 759557299618865152
Successfully queried - Tweet ID: 759447681597108224
Successfully queried - Tweet ID: 759446261539934208
Successfully queried - Tweet ID: 759197388317847553
Successfully queried - Tweet ID: 759159934323924993
Successfully queried - Tweet ID: 759099523532779520
Successfully queried - Tweet ID: 759047813560868866
Successfully queried - Tweet ID: 758854675097526272
Successfully queried - Tweet ID: 758828659922702336
Successfully queried - Tweet ID: 758740312047005698
Successfully queried - Tweet ID: 758474966123810816
Successfully queried - Tweet ID: 758467244762497024
Successfully queried - Tweet ID: 758405701903519748
Successfully queried - Tweet ID: 758355060040593408
Successfully queried - Tweet ID: 758099635764359168
Successfully queried - Tweet ID: 758041019896193024
Successfully queried - Tweet ID: 757741869644341248
Successfully queried - Tweet ID: 757729163776290825
Successfully queried - Tweet ID: 757725642876129280
Successfully queried - Tweet ID: 757611664640446465
Successfully queried - Tweet ID: 757597904299253760
Successfully queried - Tweet ID: 757596066325864448
Successfully queried - Tweet ID: 757400162377592832
Successfully queried - Tweet ID: 757393109802180609
Successfully queried - Tweet ID: 757354760399941633
Successfully queried - Tweet ID: 756998049151549440
Successfully queried - Tweet ID: 756939218950160384
Successfully queried - Tweet ID: 756651752796094464
Successfully queried - Tweet ID: 756526248105566208
Successfully queried - Tweet ID: 756303284449767430
Successfully queried - Tweet ID: 756288534030475264
Successfully queried - Tweet ID: 756275833623502848
Successfully queried - Tweet ID: 755955933503782912
Successfully queried - Tweet ID: 755206590534418437
Successfully queried - Tweet ID: 755110668769038337
Successfully queried - Tweet ID: 754874841593970688
Successfully queried - Tweet ID: 754856583969079297
Successfully queried - Tweet ID: 754747087846248448
Successfully queried - Tweet ID: 754482103782404096
Successfully queried - Tweet ID: 754449512966619136
Successfully queried - Tweet ID: 754120377874386944
Error Tweet_Id: 754011816964026368 : [{'code': 144, 'message': 'No status found with that ID.'}]
Successfully queried - Tweet ID: 754011816964026368
Successfully queried - Tweet ID: 753655901052166144
Successfully queried - Tweet ID: 753420520834629632
Successfully queried - Tweet ID: 753398408988139520
Successfully queried - Tweet ID: 753375668877008896
Successfully queried - Tweet ID: 753298634498793472
Successfully queried - Tweet ID: 753294487569522689
Successfully queried - Tweet ID: 753039830821511168
Successfully queried - Tweet ID: 753026973505581056
Successfully queried - Tweet ID: 752932432744185856
Successfully queried - Tweet ID: 752917284578922496
Successfully queried - Tweet ID: 752701944171524096
Successfully queried - Tweet ID: 752682090207055872
Successfully queried - Tweet ID: 752660715232722944
Successfully queried - Tweet ID: 752568224206688256
Successfully queried - Tweet ID: 752519690950500352
Successfully queried - Tweet ID: 752334515931054080
Successfully queried - Tweet ID: 752309394570878976
Successfully queried - Tweet ID: 752173152931807232
Successfully queried - Tweet ID: 751950017322246144
Successfully queried - Tweet ID: 751937170840121344
Successfully queried - Tweet ID: 751830394383790080
Successfully queried - Tweet ID: 751793661361422336
Successfully queried - Tweet ID: 751598357617971201
Successfully queried - Tweet ID: 751583847268179968
Successfully queried - Tweet ID: 751538714308972544
Successfully queried - Tweet ID: 751456908746354688
Successfully queried - Tweet ID: 751251247299190784
Successfully queried - Tweet ID: 751205363882532864
Successfully queried - Tweet ID: 751132876104687617
Successfully queried - Tweet ID: 750868782890057730
Successfully queried - Tweet ID: 750719632563142656
Successfully queried - Tweet ID: 750506206503038976
Successfully queried - Tweet ID: 750429297815552001
Successfully queried - Tweet ID: 750383411068534784
Successfully queried - Tweet ID: 750381685133418496
Successfully queried - Tweet ID: 750147208377409536
Successfully queried - Tweet ID: 750132105863102464
Successfully queried - Tweet ID: 750117059602808832
Successfully queried - Tweet ID: 750101899009982464
Successfully queried - Tweet ID: 750086836815486976
Successfully queried - Tweet ID: 750071704093859840
Successfully queried - Tweet ID: 750056684286914561
Successfully queried - Tweet ID: 750041628174217216
Successfully queried - Tweet ID: 750026558547456000
Successfully queried - Tweet ID: 750011400160841729
Successfully queried - Tweet ID: 749996283729883136
Successfully queried - Tweet ID: 749981277374128128
Successfully queried - Tweet ID: 749774190421639168
Successfully queried - Tweet ID: 749417653287129088
Successfully queried - Tweet ID: 749403093750648834
Successfully queried - Tweet ID: 749395845976588288
Successfully queried - Tweet ID: 749317047558017024
Successfully queried - Tweet ID: 749075273010798592
Successfully queried - Tweet ID: 749064354620928000
Successfully queried - Tweet ID: 749036806121881602
Successfully queried - Tweet ID: 748977405889503236
Successfully queried - Tweet ID: 748932637671223296
Successfully queried - Tweet ID: 748705597323898880
Successfully queried - Tweet ID: 748699167502000129
Successfully queried - Tweet ID: 748692773788876800
Successfully queried - Tweet ID: 748575535303884801
Successfully queried - Tweet ID: 748568946752774144
Successfully queried - Tweet ID: 748346686624440324
Successfully queried - Tweet ID: 748337862848962560
Successfully queried - Tweet ID: 748324050481647620
Successfully queried - Tweet ID: 748307329658011649
Successfully queried - Tweet ID: 748220828303695873
Successfully queried - Tweet ID: 747963614829678593
Successfully queried - Tweet ID: 747933425676525569
Successfully queried - Tweet ID: 747885874273214464
Successfully queried - Tweet ID: 747844099428986880
Successfully queried - Tweet ID: 747816857231626240
Successfully queried - Tweet ID: 747651430853525504
Successfully queried - Tweet ID: 747648653817413632
Successfully queried - Tweet ID: 747600769478692864
Successfully queried - Tweet ID: 747594051852075008
Successfully queried - Tweet ID: 747512671126323200
Successfully queried - Tweet ID: 747461612269887489
Successfully queried - Tweet ID: 747439450712596480
Successfully queried - Tweet ID: 747242308580548608
Successfully queried - Tweet ID: 747219827526344708
Successfully queried - Tweet ID: 747204161125646336
Successfully queried - Tweet ID: 747103485104099331
Successfully queried - Tweet ID: 746906459439529985
Successfully queried - Tweet ID: 746872823977771008
Successfully queried - Tweet ID: 746818907684614144
Successfully queried - Tweet ID: 746790600704425984
Successfully queried - Tweet ID: 746757706116112384
Successfully queried - Tweet ID: 746726898085036033
Successfully queried - Tweet ID: 746542875601690625
Successfully queried - Tweet ID: 746521445350707200
Successfully queried - Tweet ID: 746507379341139972
Successfully queried - Tweet ID: 746369468511756288
Successfully queried - Tweet ID: 746131877086527488
Successfully queried - Tweet ID: 746056683365994496
Successfully queried - Tweet ID: 745789745784041472
Successfully queried - Tweet ID: 745712589599014916
Successfully queried - Tweet ID: 745433870967832576
Successfully queried - Tweet ID: 745422732645535745
Successfully queried - Tweet ID: 745314880350101504
Successfully queried - Tweet ID: 745074613265149952
Successfully queried - Tweet ID: 745057283344719872
Successfully queried - Tweet ID: 744995568523612160
Successfully queried - Tweet ID: 744971049620602880
Successfully queried - Tweet ID: 744709971296780288
Successfully queried - Tweet ID: 744334592493166593
Successfully queried - Tweet ID: 744234799360020481
Successfully queried - Tweet ID: 744223424764059648
Successfully queried - Tweet ID: 743980027717509120
Successfully queried - Tweet ID: 743895849529389061
Successfully queried - Tweet ID: 743835915802583040
Successfully queried - Tweet ID: 743609206067040256
Successfully queried - Tweet ID: 743595368194129920
Successfully queried - Tweet ID: 743545585370791937
Successfully queried - Tweet ID: 743510151680958465
Successfully queried - Tweet ID: 743253157753532416
Successfully queried - Tweet ID: 743222593470234624
Successfully queried - Tweet ID: 743210557239623680
Successfully queried - Tweet ID: 742534281772302336
Successfully queried - Tweet ID: 742528092657332225
Successfully queried - Tweet ID: 742465774154047488
Successfully queried - Tweet ID: 742423170473463808
Successfully queried - Tweet ID: 742385895052087300
Successfully queried - Tweet ID: 742161199639494656
Successfully queried - Tweet ID: 742150209887731712
Successfully queried - Tweet ID: 741793263812808706
Successfully queried - Tweet ID: 741743634094141440
Successfully queried - Tweet ID: 741438259667034112
Successfully queried - Tweet ID: 741303864243200000
Successfully queried - Tweet ID: 741099773336379392
Successfully queried - Tweet ID: 741067306818797568
Successfully queried - Tweet ID: 740995100998766593
Successfully queried - Tweet ID: 740711788199743490
Successfully queried - Tweet ID: 740699697422163968
Successfully queried - Tweet ID: 740676976021798912
Successfully queried - Tweet ID: 740373189193256964
Successfully queried - Tweet ID: 740365076218183684
Successfully queried - Tweet ID: 740359016048689152
Successfully queried - Tweet ID: 740214038584557568
Successfully queried - Tweet ID: 739979191639244800
Successfully queried - Tweet ID: 739932936087216128
Successfully queried - Tweet ID: 739844404073074688
Successfully queried - Tweet ID: 739623569819336705
Successfully queried - Tweet ID: 739606147276148736
Successfully queried - Tweet ID: 739544079319588864
Successfully queried - Tweet ID: 739485634323156992
Successfully queried - Tweet ID: 739238157791694849
Successfully queried - Tweet ID: 738891149612572673
Successfully queried - Tweet ID: 738885046782832640
Successfully queried - Tweet ID: 738883359779196928
Successfully queried - Tweet ID: 738537504001953792
Successfully queried - Tweet ID: 738402415918125056
Successfully queried - Tweet ID: 738184450748633089
Successfully queried - Tweet ID: 738166403467907072
Successfully queried - Tweet ID: 738156290900254721
Successfully queried - Tweet ID: 737826014890496000
Successfully queried - Tweet ID: 737800304142471168
Successfully queried - Tweet ID: 737678689543020544
Successfully queried - Tweet ID: 737445876994609152
Successfully queried - Tweet ID: 737322739594330112
Successfully queried - Tweet ID: 737310737551491075
Successfully queried - Tweet ID: 736736130620620800
Successfully queried - Tweet ID: 736392552031657984
Successfully queried - Tweet ID: 736365877722001409
Successfully queried - Tweet ID: 736225175608430592
Successfully queried - Tweet ID: 736010884653420544
Successfully queried - Tweet ID: 735991953473572864
Successfully queried - Tweet ID: 735648611367784448
Successfully queried - Tweet ID: 735635087207878657
Successfully queried - Tweet ID: 735274964362878976
Successfully queried - Tweet ID: 735256018284875776
Successfully queried - Tweet ID: 735137028879360001
Successfully queried - Tweet ID: 734912297295085568
Successfully queried - Tweet ID: 734787690684657664
Successfully queried - Tweet ID: 734776360183431168
Successfully queried - Tweet ID: 734559631394082816
Successfully queried - Tweet ID: 733828123016450049
Successfully queried - Tweet ID: 733822306246479872
Successfully queried - Tweet ID: 733482008106668032
Successfully queried - Tweet ID: 733460102733135873
Successfully queried - Tweet ID: 733109485275860992
Successfully queried - Tweet ID: 732732193018155009
Successfully queried - Tweet ID: 732726085725589504
Successfully queried - Tweet ID: 732585889486888962
Successfully queried - Tweet ID: 732375214819057664
Successfully queried - Tweet ID: 732005617171337216
Successfully queried - Tweet ID: 731285275100512256
Successfully queried - Tweet ID: 731156023742988288
Successfully queried - Tweet ID: 730924654643314689
Successfully queried - Tweet ID: 730573383004487680
Successfully queried - Tweet ID: 730427201120833536
Successfully queried - Tweet ID: 730211855403241472
Successfully queried - Tweet ID: 730196704625098752
Successfully queried - Tweet ID: 729854734790754305
Successfully queried - Tweet ID: 729838605770891264
Successfully queried - Tweet ID: 729823566028484608
Successfully queried - Tweet ID: 729463711119904772
Successfully queried - Tweet ID: 729113531270991872
Successfully queried - Tweet ID: 728986383096946689
Successfully queried - Tweet ID: 728760639972315136
Successfully queried - Tweet ID: 728751179681943552
Successfully queried - Tweet ID: 728653952833728512
Successfully queried - Tweet ID: 728409960103686147
Successfully queried - Tweet ID: 728387165835677696
Successfully queried - Tweet ID: 728046963732717569
Successfully queried - Tweet ID: 728035342121635841
Successfully queried - Tweet ID: 728015554473250816
Successfully queried - Tweet ID: 727685679342333952
Successfully queried - Tweet ID: 727644517743104000
Successfully queried - Tweet ID: 727524757080539137
Successfully queried - Tweet ID: 727314416056803329
Successfully queried - Tweet ID: 727286334147182592
Successfully queried - Tweet ID: 727175381690781696
Successfully queried - Tweet ID: 727155742655025152
Successfully queried - Tweet ID: 726935089318363137
Successfully queried - Tweet ID: 726887082820554753
Successfully queried - Tweet ID: 726828223124897792
Successfully queried - Tweet ID: 726224900189511680
Successfully queried - Tweet ID: 725842289046749185
Successfully queried - Tweet ID: 725786712245440512
Successfully queried - Tweet ID: 725729321944506368
Successfully queried - Tweet ID: 725458796924002305
Successfully queried - Tweet ID: 724983749226668032
Successfully queried - Tweet ID: 724771698126512129
Successfully queried - Tweet ID: 724405726123311104
Successfully queried - Tweet ID: 724049859469295616
Successfully queried - Tweet ID: 724046343203856385
Successfully queried - Tweet ID: 724004602748780546
Successfully queried - Tweet ID: 723912936180330496
Successfully queried - Tweet ID: 723688335806480385
Successfully queried - Tweet ID: 723673163800948736
Successfully queried - Tweet ID: 723179728551723008
Successfully queried - Tweet ID: 722974582966214656
Successfully queried - Tweet ID: 722613351520608256
Successfully queried - Tweet ID: 721503162398597120
Successfully queried - Tweet ID: 721001180231503872
Successfully queried - Tweet ID: 720785406564900865
Successfully queried - Tweet ID: 720775346191278080
Successfully queried - Tweet ID: 720415127506415616
Successfully queried - Tweet ID: 720389942216527872
Successfully queried - Tweet ID: 720340705894408192
Successfully queried - Tweet ID: 720059472081784833
Successfully queried - Tweet ID: 720043174954147842
Successfully queried - Tweet ID: 719991154352222208
Successfully queried - Tweet ID: 719704490224398336
Successfully queried - Tweet ID: 719551379208073216
Successfully queried - Tweet ID: 719367763014393856
Successfully queried - Tweet ID: 719339463458033665
Successfully queried - Tweet ID: 719332531645071360
Successfully queried - Tweet ID: 718971898235854848
Successfully queried - Tweet ID: 718939241951195136
Successfully queried - Tweet ID: 718631497683582976
Successfully queried - Tweet ID: 718613305783398402
Successfully queried - Tweet ID: 718540630683709445
Successfully queried - Tweet ID: 718460005985447936
Successfully queried - Tweet ID: 718454725339934721
Successfully queried - Tweet ID: 718246886998687744
Successfully queried - Tweet ID: 718234618122661888
Successfully queried - Tweet ID: 717841801130979328
Successfully queried - Tweet ID: 717790033953034240
Successfully queried - Tweet ID: 717537687239008257
Successfully queried - Tweet ID: 717428917016076293
Successfully queried - Tweet ID: 717421804990701568
Successfully queried - Tweet ID: 717047459982213120
Successfully queried - Tweet ID: 717009362452090881
Successfully queried - Tweet ID: 716802964044845056
Successfully queried - Tweet ID: 716791146589110272
Successfully queried - Tweet ID: 716730379797970944
Successfully queried - Tweet ID: 716447146686459905
Successfully queried - Tweet ID: 716439118184652801
Successfully queried - Tweet ID: 716285507865542656
Successfully queried - Tweet ID: 716080869887381504
Successfully queried - Tweet ID: 715928423106027520
Successfully queried - Tweet ID: 715758151270801409
Successfully queried - Tweet ID: 715733265223708672
Successfully queried - Tweet ID: 715704790270025728
Successfully queried - Tweet ID: 715696743237730304
Successfully queried - Tweet ID: 715680795826982913
Successfully queried - Tweet ID: 715360349751484417
Successfully queried - Tweet ID: 715342466308784130
Successfully queried - Tweet ID: 715220193576927233
Successfully queried - Tweet ID: 715200624753819648
Successfully queried - Tweet ID: 715009755312439296
Successfully queried - Tweet ID: 714982300363173890
Successfully queried - Tweet ID: 714962719905021952
Successfully queried - Tweet ID: 714957620017307648
Successfully queried - Tweet ID: 714631576617938945
Successfully queried - Tweet ID: 714606013974974464
Successfully queried - Tweet ID: 714485234495041536
Successfully queried - Tweet ID: 714258258790387713
Successfully queried - Tweet ID: 714251586676113411
Successfully queried - Tweet ID: 714214115368108032
Successfully queried - Tweet ID: 714141408463036416
Successfully queried - Tweet ID: 713919462244790272
Successfully queried - Tweet ID: 713909862279876608
Successfully queried - Tweet ID: 713900603437621249
Successfully queried - Tweet ID: 713761197720473600
Successfully queried - Tweet ID: 713411074226274305
Successfully queried - Tweet ID: 713177543487135744
Successfully queried - Tweet ID: 713175907180089344
Successfully queried - Tweet ID: 712809025985978368
Successfully queried - Tweet ID: 712717840512598017
Successfully queried - Tweet ID: 712668654853337088
Successfully queried - Tweet ID: 712438159032893441
Successfully queried - Tweet ID: 712309440758808576
Successfully queried - Tweet ID: 712097430750289920
Successfully queried - Tweet ID: 712092745624633345
Successfully queried - Tweet ID: 712085617388212225
Successfully queried - Tweet ID: 712065007010385924
Successfully queried - Tweet ID: 711998809858043904
Successfully queried - Tweet ID: 711968124745228288
Successfully queried - Tweet ID: 711743778164514816
Successfully queried - Tweet ID: 711732680602345472
Successfully queried - Tweet ID: 711694788429553666
Successfully queried - Tweet ID: 711652651650457602
Successfully queried - Tweet ID: 711363825979756544
Successfully queried - Tweet ID: 711306686208872448
Successfully queried - Tweet ID: 711008018775851008
Successfully queried - Tweet ID: 710997087345876993
Successfully queried - Tweet ID: 710844581445812225
Successfully queried - Tweet ID: 710833117892898816
Successfully queried - Tweet ID: 710658690886586372
Successfully queried - Tweet ID: 710609963652087808
Successfully queried - Tweet ID: 710588934686908417
Successfully queried - Tweet ID: 710296729921429505
Successfully queried - Tweet ID: 710283270106132480
Successfully queried - Tweet ID: 710272297844797440
Successfully queried - Tweet ID: 710269109699739648
Successfully queried - Tweet ID: 710153181850935296
Successfully queried - Tweet ID: 710140971284037632
Successfully queried - Tweet ID: 710117014656950272
Successfully queried - Tweet ID: 709918798883774466
Successfully queried - Tweet ID: 709901256215666688
Successfully queried - Tweet ID: 709852847387627521
Successfully queried - Tweet ID: 709566166965075968
Successfully queried - Tweet ID: 709556954897764353
Successfully queried - Tweet ID: 709519240576036864
Successfully queried - Tweet ID: 709449600415961088
Successfully queried - Tweet ID: 709409458133323776
Successfully queried - Tweet ID: 709225125749587968
Successfully queried - Tweet ID: 709207347839836162
Successfully queried - Tweet ID: 709198395643068416
Successfully queried - Tweet ID: 709179584944730112
Successfully queried - Tweet ID: 709158332880297985
Successfully queried - Tweet ID: 709042156699303936
Successfully queried - Tweet ID: 708853462201716736
Successfully queried - Tweet ID: 708845821941387268
Successfully queried - Tweet ID: 708834316713893888
Successfully queried - Tweet ID: 708810915978854401
Successfully queried - Tweet ID: 708738143638450176
Successfully queried - Tweet ID: 708711088997666817
Successfully queried - Tweet ID: 708479650088034305
Successfully queried - Tweet ID: 708469915515297792
Successfully queried - Tweet ID: 708400866336894977
Successfully queried - Tweet ID: 708356463048204288
Successfully queried - Tweet ID: 708349470027751425
Successfully queried - Tweet ID: 708149363256774660
Successfully queried - Tweet ID: 708130923141795840
Successfully queried - Tweet ID: 708119489313951744
Successfully queried - Tweet ID: 708109389455101952
Successfully queried - Tweet ID: 708026248782585858
Successfully queried - Tweet ID: 707995814724026368
Successfully queried - Tweet ID: 707983188426153984
Successfully queried - Tweet ID: 707969809498152960
Successfully queried - Tweet ID: 707776935007539200
Successfully queried - Tweet ID: 707741517457260545
Successfully queried - Tweet ID: 707738799544082433
Successfully queried - Tweet ID: 707693576495472641
Successfully queried - Tweet ID: 707629649552134146
Successfully queried - Tweet ID: 707610948723478529
Successfully queried - Tweet ID: 707420581654872064
Successfully queried - Tweet ID: 707411934438625280
Successfully queried - Tweet ID: 707387676719185920
Successfully queried - Tweet ID: 707377100785885184
Successfully queried - Tweet ID: 707315916783140866
Successfully queried - Tweet ID: 707297311098011648
Successfully queried - Tweet ID: 707059547140169728
Successfully queried - Tweet ID: 707038192327901184
Successfully queried - Tweet ID: 707021089608753152
Successfully queried - Tweet ID: 707014260413456384
Successfully queried - Tweet ID: 706904523814649856
Successfully queried - Tweet ID: 706901761596989440
Successfully queried - Tweet ID: 706681918348251136
Successfully queried - Tweet ID: 706644897839910912
Successfully queried - Tweet ID: 706593038911545345
Successfully queried - Tweet ID: 706538006853918722
Successfully queried - Tweet ID: 706516534877929472
Successfully queried - Tweet ID: 706346369204748288
Successfully queried - Tweet ID: 706310011488698368
Successfully queried - Tweet ID: 706291001778950144
Successfully queried - Tweet ID: 706265994973601792
Successfully queried - Tweet ID: 706169069255446529
Successfully queried - Tweet ID: 706166467411222528
Successfully queried - Tweet ID: 706153300320784384
Successfully queried - Tweet ID: 705975130514706432
Successfully queried - Tweet ID: 705970349788291072
Successfully queried - Tweet ID: 705898680587526145
Successfully queried - Tweet ID: 705786532653883392
Successfully queried - Tweet ID: 705591895322394625
Successfully queried - Tweet ID: 705475953783398401
Successfully queried - Tweet ID: 705442520700944385
Successfully queried - Tweet ID: 705428427625635840
Successfully queried - Tweet ID: 705239209544720384
Successfully queried - Tweet ID: 705223444686888960
Successfully queried - Tweet ID: 705102439679201280
Successfully queried - Tweet ID: 705066031337840642
Successfully queried - Tweet ID: 704871453724954624
Successfully queried - Tweet ID: 704859558691414016
Successfully queried - Tweet ID: 704847917308362754
Successfully queried - Tweet ID: 704819833553219584
Successfully queried - Tweet ID: 704761120771465216
Successfully queried - Tweet ID: 704499785726889984
Successfully queried - Tweet ID: 704491224099647488
Successfully queried - Tweet ID: 704480331685040129
Successfully queried - Tweet ID: 704364645503647744
Successfully queried - Tweet ID: 704347321748819968
Successfully queried - Tweet ID: 704134088924532736
Successfully queried - Tweet ID: 704113298707505153
Successfully queried - Tweet ID: 704054845121142784
Successfully queried - Tweet ID: 703774238772166656
Successfully queried - Tweet ID: 703769065844768768
Successfully queried - Tweet ID: 703631701117943808
Successfully queried - Tweet ID: 703611486317502464
Successfully queried - Tweet ID: 703425003149250560
Successfully queried - Tweet ID: 703407252292673536
Successfully queried - Tweet ID: 703382836347330562
Successfully queried - Tweet ID: 703356393781329922
Successfully queried - Tweet ID: 703268521220972544
Successfully queried - Tweet ID: 703079050210877440
Successfully queried - Tweet ID: 703041949650034688
Successfully queried - Tweet ID: 702932127499816960
Successfully queried - Tweet ID: 702899151802126337
Successfully queried - Tweet ID: 702684942141153280
Successfully queried - Tweet ID: 702671118226825216
Successfully queried - Tweet ID: 702598099714314240
Successfully queried - Tweet ID: 702539513671897089
Successfully queried - Tweet ID: 702332542343577600
Successfully queried - Tweet ID: 702321140488925184
Successfully queried - Tweet ID: 702276748847800320
Successfully queried - Tweet ID: 702217446468493312
Successfully queried - Tweet ID: 701981390485725185
Successfully queried - Tweet ID: 701952816642965504
Successfully queried - Tweet ID: 701889187134500865
Successfully queried - Tweet ID: 701805642395348998
Successfully queried - Tweet ID: 701601587219795968
Successfully queried - Tweet ID: 701570477911896070
Successfully queried - Tweet ID: 701545186879471618
Successfully queried - Tweet ID: 701214700881756160
Successfully queried - Tweet ID: 700890391244103680
Successfully queried - Tweet ID: 700864154249383937
Successfully queried - Tweet ID: 700847567345688576
Successfully queried - Tweet ID: 700796979434098688
Successfully queried - Tweet ID: 700747788515020802
Successfully queried - Tweet ID: 700518061187723268
Successfully queried - Tweet ID: 700505138482569216
Successfully queried - Tweet ID: 700462010979500032
Successfully queried - Tweet ID: 700167517596164096
Successfully queried - Tweet ID: 700151421916807169
Successfully queried - Tweet ID: 700143752053182464
Successfully queried - Tweet ID: 700062718104104960
Successfully queried - Tweet ID: 700029284593901568
Successfully queried - Tweet ID: 700002074055016451
Successfully queried - Tweet ID: 699801817392291840
Successfully queried - Tweet ID: 699788877217865730
Successfully queried - Tweet ID: 699779630832685056
Successfully queried - Tweet ID: 699775878809702401
Successfully queried - Tweet ID: 699691744225525762
Successfully queried - Tweet ID: 699446877801091073
Successfully queried - Tweet ID: 699434518667751424
Successfully queried - Tweet ID: 699423671849451520
Successfully queried - Tweet ID: 699413908797464576
Successfully queried - Tweet ID: 699370870310113280
Successfully queried - Tweet ID: 699323444782047232
Successfully queried - Tweet ID: 699088579889332224
Successfully queried - Tweet ID: 699079609774645248
Successfully queried - Tweet ID: 699072405256409088
Successfully queried - Tweet ID: 699060279947165696
Successfully queried - Tweet ID: 699036661657767936
Successfully queried - Tweet ID: 698989035503689728
Successfully queried - Tweet ID: 698953797952008193
Successfully queried - Tweet ID: 698907974262222848
Successfully queried - Tweet ID: 698710712454139905
Successfully queried - Tweet ID: 698703483621523456
Successfully queried - Tweet ID: 698635131305795584
Successfully queried - Tweet ID: 698549713696649216
Successfully queried - Tweet ID: 698355670425473025
Successfully queried - Tweet ID: 698342080612007937
Successfully queried - Tweet ID: 698262614669991936
Successfully queried - Tweet ID: 698195409219559425
Successfully queried - Tweet ID: 698178924120031232
Successfully queried - Tweet ID: 697995514407682048
Successfully queried - Tweet ID: 697990423684476929
Successfully queried - Tweet ID: 697943111201378304
Successfully queried - Tweet ID: 697881462549430272
Successfully queried - Tweet ID: 697630435728322560
Successfully queried - Tweet ID: 697616773278015490
Successfully queried - Tweet ID: 697596423848730625
Successfully queried - Tweet ID: 697575480820686848
Successfully queried - Tweet ID: 697516214579523584
Successfully queried - Tweet ID: 697482927769255936
Successfully queried - Tweet ID: 697463031882764288
Successfully queried - Tweet ID: 697270446429966336
Successfully queried - Tweet ID: 697259378236399616
Successfully queried - Tweet ID: 697255105972801536
Successfully queried - Tweet ID: 697242256848379904
Successfully queried - Tweet ID: 696900204696625153
Successfully queried - Tweet ID: 696894894812565505
Successfully queried - Tweet ID: 696886256886657024
Successfully queried - Tweet ID: 696877980375769088
Successfully queried - Tweet ID: 696754882863349760
Successfully queried - Tweet ID: 696744641916489729
Successfully queried - Tweet ID: 696713835009417216
Successfully queried - Tweet ID: 696518437233913856
Successfully queried - Tweet ID: 696490539101908992
Successfully queried - Tweet ID: 696488710901260288
Successfully queried - Tweet ID: 696405997980676096
Successfully queried - Tweet ID: 696100768806522880
Successfully queried - Tweet ID: 695816827381944320
Successfully queried - Tweet ID: 695794761660297217
Successfully queried - Tweet ID: 695767669421768709
Successfully queried - Tweet ID: 695629776980148225
Successfully queried - Tweet ID: 695446424020918272
Successfully queried - Tweet ID: 695409464418041856
Successfully queried - Tweet ID: 695314793360662529
Successfully queried - Tweet ID: 695095422348574720
Successfully queried - Tweet ID: 695074328191332352
Successfully queried - Tweet ID: 695064344191721472
Successfully queried - Tweet ID: 695051054296211456
Successfully queried - Tweet ID: 694925794720792577
Successfully queried - Tweet ID: 694905863685980160
Successfully queried - Tweet ID: 694669722378485760
Successfully queried - Tweet ID: 694356675654983680
Successfully queried - Tweet ID: 694352839993344000
Successfully queried - Tweet ID: 694342028726001664
Successfully queried - Tweet ID: 694329668942569472
Successfully queried - Tweet ID: 694206574471057408
Successfully queried - Tweet ID: 694183373896572928
Successfully queried - Tweet ID: 694001791655137281
Successfully queried - Tweet ID: 693993230313091072
Successfully queried - Tweet ID: 693942351086120961
Successfully queried - Tweet ID: 693647888581312512
Successfully queried - Tweet ID: 693644216740769793
Successfully queried - Tweet ID: 693642232151285760
Successfully queried - Tweet ID: 693629975228977152
Successfully queried - Tweet ID: 693622659251335168
Successfully queried - Tweet ID: 693590843962331137
Successfully queried - Tweet ID: 693582294167244802
Successfully queried - Tweet ID: 693486665285931008
Successfully queried - Tweet ID: 693280720173801472
Successfully queried - Tweet ID: 693267061318012928
Successfully queried - Tweet ID: 693262851218264065
Successfully queried - Tweet ID: 693231807727280129
Successfully queried - Tweet ID: 693155686491000832
Successfully queried - Tweet ID: 693109034023534592
Successfully queried - Tweet ID: 693095443459342336
Successfully queried - Tweet ID: 692919143163629568
Successfully queried - Tweet ID: 692905862751522816
Successfully queried - Tweet ID: 692901601640583168
Successfully queried - Tweet ID: 692894228850999298
Successfully queried - Tweet ID: 692828166163931137
Successfully queried - Tweet ID: 692752401762250755
Successfully queried - Tweet ID: 692568918515392513
Successfully queried - Tweet ID: 692535307825213440
Successfully queried - Tweet ID: 692530551048294401
Successfully queried - Tweet ID: 692423280028966913
Successfully queried - Tweet ID: 692417313023332352
Successfully queried - Tweet ID: 692187005137076224
Successfully queried - Tweet ID: 692158366030913536
Successfully queried - Tweet ID: 692142790915014657
Successfully queried - Tweet ID: 692041934689402880
Successfully queried - Tweet ID: 692017291282812928
Successfully queried - Tweet ID: 691820333922455552
Successfully queried - Tweet ID: 691793053716221953
Successfully queried - Tweet ID: 691756958957883396
Successfully queried - Tweet ID: 691675652215414786
Successfully queried - Tweet ID: 691483041324204033
Successfully queried - Tweet ID: 691459709405118465
Successfully queried - Tweet ID: 691444869282295808
Successfully queried - Tweet ID: 691416866452082688
Successfully queried - Tweet ID: 691321916024623104
Successfully queried - Tweet ID: 691096613310316544
Successfully queried - Tweet ID: 691090071332753408
Successfully queried - Tweet ID: 690989312272396288
Successfully queried - Tweet ID: 690959652130045952
Successfully queried - Tweet ID: 690938899477221376
Successfully queried - Tweet ID: 690932576555528194
Successfully queried - Tweet ID: 690735892932222976
Successfully queried - Tweet ID: 690728923253055490
Successfully queried - Tweet ID: 690690673629138944
Successfully queried - Tweet ID: 690649993829576704
Successfully queried - Tweet ID: 690607260360429569
Successfully queried - Tweet ID: 690597161306841088
Successfully queried - Tweet ID: 690400367696297985
Successfully queried - Tweet ID: 690374419777196032
Successfully queried - Tweet ID: 690360449368465409
Successfully queried - Tweet ID: 690348396616552449
Successfully queried - Tweet ID: 690248561355657216
Successfully queried - Tweet ID: 690021994562220032
Successfully queried - Tweet ID: 690015576308211712
Successfully queried - Tweet ID: 690005060500217858
Successfully queried - Tweet ID: 689999384604450816
Successfully queried - Tweet ID: 689993469801164801
Successfully queried - Tweet ID: 689977555533848577
Successfully queried - Tweet ID: 689905486972461056
Successfully queried - Tweet ID: 689877686181715968
Successfully queried - Tweet ID: 689835978131935233
Successfully queried - Tweet ID: 689661964914655233
Successfully queried - Tweet ID: 689659372465688576
Successfully queried - Tweet ID: 689623661272240129
Successfully queried - Tweet ID: 689599056876867584
Successfully queried - Tweet ID: 689557536375177216
Successfully queried - Tweet ID: 689517482558820352
Successfully queried - Tweet ID: 689289219123089408
Successfully queried - Tweet ID: 689283819090870273
Successfully queried - Tweet ID: 689280876073582592
Successfully queried - Tweet ID: 689275259254616065
Successfully queried - Tweet ID: 689255633275777024
Successfully queried - Tweet ID: 689154315265683456
Successfully queried - Tweet ID: 689143371370250240
Successfully queried - Tweet ID: 688916208532455424
Successfully queried - Tweet ID: 688908934925697024
Successfully queried - Tweet ID: 688898160958271489
Successfully queried - Tweet ID: 688894073864884227
Successfully queried - Tweet ID: 688828561667567616
Successfully queried - Tweet ID: 688804835492233216
Successfully queried - Tweet ID: 688789766343622656
Successfully queried - Tweet ID: 688547210804498433
Successfully queried - Tweet ID: 688519176466644993
Successfully queried - Tweet ID: 688385280030670848
Successfully queried - Tweet ID: 688211956440801280
Successfully queried - Tweet ID: 688179443353796608
Successfully queried - Tweet ID: 688116655151435777
Successfully queried - Tweet ID: 688064179421470721
Successfully queried - Tweet ID: 687841446767013888
Successfully queried - Tweet ID: 687826841265172480
Successfully queried - Tweet ID: 687818504314159109
Successfully queried - Tweet ID: 687807801670897665
Successfully queried - Tweet ID: 687732144991551489
Successfully queried - Tweet ID: 687704180304273409
Successfully queried - Tweet ID: 687664829264453632
Successfully queried - Tweet ID: 687494652870668288
Successfully queried - Tweet ID: 687480748861947905
Successfully queried - Tweet ID: 687476254459715584
Successfully queried - Tweet ID: 687460506001633280
Successfully queried - Tweet ID: 687399393394311168
Successfully queried - Tweet ID: 687317306314240000
Successfully queried - Tweet ID: 687312378585812992
Successfully queried - Tweet ID: 687127927494963200
Successfully queried - Tweet ID: 687124485711986689
Successfully queried - Tweet ID: 687109925361856513
Successfully queried - Tweet ID: 687102708889812993
Successfully queried - Tweet ID: 687096057537363968
Successfully queried - Tweet ID: 686947101016735744
Successfully queried - Tweet ID: 686760001961103360
Successfully queried - Tweet ID: 686749460672679938
Successfully queried - Tweet ID: 686730991906516992
Successfully queried - Tweet ID: 686683045143953408
Successfully queried - Tweet ID: 686618349602762752
Successfully queried - Tweet ID: 686606069955735556
Successfully queried - Tweet ID: 686394059078897668
Successfully queried - Tweet ID: 686386521809772549
Successfully queried - Tweet ID: 686377065986265092
Successfully queried - Tweet ID: 686358356425093120
Successfully queried - Tweet ID: 686286779679375361
Successfully queried - Tweet ID: 686050296934563840
Successfully queried - Tweet ID: 686035780142297088
Successfully queried - Tweet ID: 686034024800862208
Successfully queried - Tweet ID: 686007916130873345
Successfully queried - Tweet ID: 686003207160610816
Successfully queried - Tweet ID: 685973236358713344
Successfully queried - Tweet ID: 685943807276412928
Successfully queried - Tweet ID: 685906723014619143
Successfully queried - Tweet ID: 685681090388975616
Successfully queried - Tweet ID: 685667379192414208
Successfully queried - Tweet ID: 685663452032069632
Successfully queried - Tweet ID: 685641971164143616
Successfully queried - Tweet ID: 685547936038666240
Successfully queried - Tweet ID: 685532292383666176
Successfully queried - Tweet ID: 685325112850124800
Successfully queried - Tweet ID: 685321586178670592
Successfully queried - Tweet ID: 685315239903100929
Successfully queried - Tweet ID: 685307451701334016
Successfully queried - Tweet ID: 685268753634967552
Successfully queried - Tweet ID: 685198997565345792
Successfully queried - Tweet ID: 685169283572338688
Successfully queried - Tweet ID: 684969860808454144
Successfully queried - Tweet ID: 684959798585110529
Successfully queried - Tweet ID: 684940049151070208
Successfully queried - Tweet ID: 684926975086034944
Successfully queried - Tweet ID: 684914660081053696
Successfully queried - Tweet ID: 684902183876321280
Successfully queried - Tweet ID: 684880619965411328
Successfully queried - Tweet ID: 684830982659280897
Successfully queried - Tweet ID: 684800227459624960
Successfully queried - Tweet ID: 684594889858887680
Successfully queried - Tweet ID: 684588130326986752
Successfully queried - Tweet ID: 684567543613382656
Successfully queried - Tweet ID: 684538444857667585
Successfully queried - Tweet ID: 684481074559381504
Successfully queried - Tweet ID: 684460069371654144
Successfully queried - Tweet ID: 684241637099323392
Successfully queried - Tweet ID: 684225744407494656
Successfully queried - Tweet ID: 684222868335505415
Successfully queried - Tweet ID: 684200372118904832
Successfully queried - Tweet ID: 684195085588783105
Successfully queried - Tweet ID: 684188786104872960
Successfully queried - Tweet ID: 684177701129875456
Successfully queried - Tweet ID: 684147889187209216
Successfully queried - Tweet ID: 684122891630342144
Successfully queried - Tweet ID: 684097758874210310
Successfully queried - Tweet ID: 683857920510050305
Successfully queried - Tweet ID: 683852578183077888
Successfully queried - Tweet ID: 683849932751646720
Successfully queried - Tweet ID: 683834909291606017
Successfully queried - Tweet ID: 683828599284170753
Successfully queried - Tweet ID: 683773439333797890
Successfully queried - Tweet ID: 683742671509258241
Successfully queried - Tweet ID: 683515932363329536
Successfully queried - Tweet ID: 683498322573824003
Successfully queried - Tweet ID: 683481228088049664
Successfully queried - Tweet ID: 683462770029932544
Successfully queried - Tweet ID: 683449695444799489
Successfully queried - Tweet ID: 683391852557561860
Successfully queried - Tweet ID: 683357973142474752
Successfully queried - Tweet ID: 683142553609318400
Successfully queried - Tweet ID: 683111407806746624

Rate limit reached. Sleeping for: 571

Successfully queried - Tweet ID: 683098815881154561
Successfully queried - Tweet ID: 683078886620553216
Successfully queried - Tweet ID: 683030066213818368
Successfully queried - Tweet ID: 682962037429899265
Successfully queried - Tweet ID: 682808988178739200
Successfully queried - Tweet ID: 682788441537560576
Successfully queried - Tweet ID: 682750546109968385
Successfully queried - Tweet ID: 682697186228989953
Successfully queried - Tweet ID: 682662431982772225
Successfully queried - Tweet ID: 682638830361513985
Successfully queried - Tweet ID: 682429480204398592
Successfully queried - Tweet ID: 682406705142087680
Successfully queried - Tweet ID: 682393905736888321
Successfully queried - Tweet ID: 682389078323662849
Successfully queried - Tweet ID: 682303737705140231
Successfully queried - Tweet ID: 682259524040966145
Successfully queried - Tweet ID: 682242692827447297
Successfully queried - Tweet ID: 682088079302213632
Successfully queried - Tweet ID: 682059653698686977
Successfully queried - Tweet ID: 682047327939461121
Successfully queried - Tweet ID: 682032003584274432
Successfully queried - Tweet ID: 682003177596559360
Successfully queried - Tweet ID: 681981167097122816
Successfully queried - Tweet ID: 681891461017812993
Successfully queried - Tweet ID: 681694085539872773
Successfully queried - Tweet ID: 681679526984871937
Successfully queried - Tweet ID: 681654059175129088
Successfully queried - Tweet ID: 681610798867845120
Successfully queried - Tweet ID: 681579835668455424
Successfully queried - Tweet ID: 681523177663676416
Successfully queried - Tweet ID: 681340665377193984
Successfully queried - Tweet ID: 681339448655802368
Successfully queried - Tweet ID: 681320187870711809
Successfully queried - Tweet ID: 681302363064414209
Successfully queried - Tweet ID: 681297372102656000
Successfully queried - Tweet ID: 681281657291280384
Successfully queried - Tweet ID: 681261549936340994
Successfully queried - Tweet ID: 681242418453299201
Successfully queried - Tweet ID: 681231109724700672
Successfully queried - Tweet ID: 681193455364796417
Successfully queried - Tweet ID: 680970795137544192
Successfully queried - Tweet ID: 680959110691590145
Successfully queried - Tweet ID: 680940246314430465
Successfully queried - Tweet ID: 680934982542561280
Successfully queried - Tweet ID: 680913438424612864
Successfully queried - Tweet ID: 680889648562991104
Successfully queried - Tweet ID: 680836378243002368
Successfully queried - Tweet ID: 680805554198020098
Successfully queried - Tweet ID: 680801747103793152
Successfully queried - Tweet ID: 680798457301471234
Successfully queried - Tweet ID: 680609293079592961
Successfully queried - Tweet ID: 680583894916304897
Successfully queried - Tweet ID: 680497766108381184
Successfully queried - Tweet ID: 680494726643068929
Successfully queried - Tweet ID: 680473011644985345
Successfully queried - Tweet ID: 680440374763077632
Successfully queried - Tweet ID: 680221482581123072
Successfully queried - Tweet ID: 680206703334408192
Successfully queried - Tweet ID: 680191257256136705
Successfully queried - Tweet ID: 680176173301628928
Successfully queried - Tweet ID: 680161097740095489
Successfully queried - Tweet ID: 680145970311643136
Successfully queried - Tweet ID: 680130881361686529
Successfully queried - Tweet ID: 680115823365742593
Successfully queried - Tweet ID: 680100725817409536
Successfully queried - Tweet ID: 680085611152338944
Successfully queried - Tweet ID: 680070545539371008
Error Tweet_Id: 680055455951884288 : [{'code': 144, 'message': 'No status found with that ID.'}]
Successfully queried - Tweet ID: 680055455951884288
Successfully queried - Tweet ID: 679877062409191424
Successfully queried - Tweet ID: 679872969355714560
Successfully queried - Tweet ID: 679862121895714818
Successfully queried - Tweet ID: 679854723806179328
Successfully queried - Tweet ID: 679844490799091713
Successfully queried - Tweet ID: 679828447187857408
Successfully queried - Tweet ID: 679777920601223168
Successfully queried - Tweet ID: 679736210798047232
Successfully queried - Tweet ID: 679729593985699840
Successfully queried - Tweet ID: 679722016581222400
Successfully queried - Tweet ID: 679530280114372609
Successfully queried - Tweet ID: 679527802031484928
Successfully queried - Tweet ID: 679511351870550016
Successfully queried - Tweet ID: 679503373272485890
Successfully queried - Tweet ID: 679475951516934144
Successfully queried - Tweet ID: 679462823135686656
Successfully queried - Tweet ID: 679405845277462528
Successfully queried - Tweet ID: 679158373988876288
Successfully queried - Tweet ID: 679148763231985668
Successfully queried - Tweet ID: 679132435750195208
Successfully queried - Tweet ID: 679111216690831360
Successfully queried - Tweet ID: 679062614270468097
Successfully queried - Tweet ID: 679047485189439488
Successfully queried - Tweet ID: 679001094530465792
Successfully queried - Tweet ID: 678991772295516161
Successfully queried - Tweet ID: 678969228704284672
Successfully queried - Tweet ID: 678800283649069056
Successfully queried - Tweet ID: 678798276842360832
Successfully queried - Tweet ID: 678774928607469569
Successfully queried - Tweet ID: 678767140346941444
Successfully queried - Tweet ID: 678764513869611008
Successfully queried - Tweet ID: 678755239630127104
Successfully queried - Tweet ID: 678740035362037760
Successfully queried - Tweet ID: 678708137298427904
Successfully queried - Tweet ID: 678675843183484930
Successfully queried - Tweet ID: 678643457146150913
Successfully queried - Tweet ID: 678446151570427904
Successfully queried - Tweet ID: 678424312106393600
Successfully queried - Tweet ID: 678410210315247616
Successfully queried - Tweet ID: 678399652199309312
Successfully queried - Tweet ID: 678396796259975168
Successfully queried - Tweet ID: 678389028614488064
Successfully queried - Tweet ID: 678380236862578688
Successfully queried - Tweet ID: 678341075375947776
Successfully queried - Tweet ID: 678334497360859136
Successfully queried - Tweet ID: 678278586130948096
Successfully queried - Tweet ID: 678255464182861824
Successfully queried - Tweet ID: 678023323247357953
Successfully queried - Tweet ID: 678021115718029313
Successfully queried - Tweet ID: 677961670166224897
Successfully queried - Tweet ID: 677918531514703872
Successfully queried - Tweet ID: 677895101218201600
Successfully queried - Tweet ID: 677716515794329600
Successfully queried - Tweet ID: 677700003327029250
Successfully queried - Tweet ID: 677698403548192770
Successfully queried - Tweet ID: 677687604918272002
Successfully queried - Tweet ID: 677673981332312066
Successfully queried - Tweet ID: 677662372920729601
Successfully queried - Tweet ID: 677644091929329666
Successfully queried - Tweet ID: 677573743309385728
Successfully queried - Tweet ID: 677565715327688705
Successfully queried - Tweet ID: 677557565589463040
Successfully queried - Tweet ID: 677547928504967168
Successfully queried - Tweet ID: 677530072887205888
Successfully queried - Tweet ID: 677335745548390400
Successfully queried - Tweet ID: 677334615166730240
Successfully queried - Tweet ID: 677331501395156992
Successfully queried - Tweet ID: 677328882937298944
Successfully queried - Tweet ID: 677314812125323265
Successfully queried - Tweet ID: 677301033169788928
Successfully queried - Tweet ID: 677269281705472000
Successfully queried - Tweet ID: 677228873407442944
Successfully queried - Tweet ID: 677187300187611136
Successfully queried - Tweet ID: 676975532580409345
Successfully queried - Tweet ID: 676957860086095872
Successfully queried - Tweet ID: 676949632774234114
Successfully queried - Tweet ID: 676948236477857792
Successfully queried - Tweet ID: 676946864479084545
Successfully queried - Tweet ID: 676942428000112642
Successfully queried - Tweet ID: 676936541936185344
Successfully queried - Tweet ID: 676916996760600576
Successfully queried - Tweet ID: 676897532954456065
Successfully queried - Tweet ID: 676864501615042560
Successfully queried - Tweet ID: 676821958043033607
Successfully queried - Tweet ID: 676819651066732545
Successfully queried - Tweet ID: 676811746707918848
Successfully queried - Tweet ID: 676776431406465024
Successfully queried - Tweet ID: 676617503762681856
Successfully queried - Tweet ID: 676613908052996102
Successfully queried - Tweet ID: 676606785097199616
Successfully queried - Tweet ID: 676603393314578432
Successfully queried - Tweet ID: 676593408224403456
Successfully queried - Tweet ID: 676590572941893632
Successfully queried - Tweet ID: 676588346097852417
Successfully queried - Tweet ID: 676582956622721024
Successfully queried - Tweet ID: 676575501977128964
Successfully queried - Tweet ID: 676533798876651520
Successfully queried - Tweet ID: 676496375194980353
Successfully queried - Tweet ID: 676470639084101634
Successfully queried - Tweet ID: 676440007570247681
Successfully queried - Tweet ID: 676430933382295552
Successfully queried - Tweet ID: 676263575653122048
Successfully queried - Tweet ID: 676237365392908289
Successfully queried - Tweet ID: 676219687039057920
Successfully queried - Tweet ID: 676215927814406144
Successfully queried - Tweet ID: 676191832485810177
Successfully queried - Tweet ID: 676146341966438401
Successfully queried - Tweet ID: 676121918416756736
Successfully queried - Tweet ID: 676101918813499392
Successfully queried - Tweet ID: 676098748976615425
Successfully queried - Tweet ID: 676089483918516224
Successfully queried - Tweet ID: 675898130735476737
Successfully queried - Tweet ID: 675891555769696257
Successfully queried - Tweet ID: 675888385639251968
Successfully queried - Tweet ID: 675878199931371520
Successfully queried - Tweet ID: 675870721063669760
Successfully queried - Tweet ID: 675853064436391936
Successfully queried - Tweet ID: 675849018447167488
Successfully queried - Tweet ID: 675845657354215424
Successfully queried - Tweet ID: 675822767435051008
Successfully queried - Tweet ID: 675820929667219457
Successfully queried - Tweet ID: 675798442703122432
Successfully queried - Tweet ID: 675781562965868544
Successfully queried - Tweet ID: 675740360753160193
Successfully queried - Tweet ID: 675710890956750848
Successfully queried - Tweet ID: 675707330206547968
Successfully queried - Tweet ID: 675706639471788032
Successfully queried - Tweet ID: 675534494439489536
Successfully queried - Tweet ID: 675531475945709568
Successfully queried - Tweet ID: 675522403582218240
Successfully queried - Tweet ID: 675517828909424640
Successfully queried - Tweet ID: 675501075957489664
Successfully queried - Tweet ID: 675497103322386432
Successfully queried - Tweet ID: 675489971617296384
Successfully queried - Tweet ID: 675483430902214656
Successfully queried - Tweet ID: 675432746517426176
Successfully queried - Tweet ID: 675372240448454658
Successfully queried - Tweet ID: 675362609739206656
Successfully queried - Tweet ID: 675354435921575936
Successfully queried - Tweet ID: 675349384339542016
Successfully queried - Tweet ID: 675334060156301312
Successfully queried - Tweet ID: 675166823650848770
Successfully queried - Tweet ID: 675153376133427200
Successfully queried - Tweet ID: 675149409102012420
Successfully queried - Tweet ID: 675147105808306176
Successfully queried - Tweet ID: 675146535592706048
Successfully queried - Tweet ID: 675145476954566656
Successfully queried - Tweet ID: 675135153782571009
Successfully queried - Tweet ID: 675113801096802304
Successfully queried - Tweet ID: 675111688094527488
Successfully queried - Tweet ID: 675109292475830276
Successfully queried - Tweet ID: 675047298674663426
Successfully queried - Tweet ID: 675015141583413248
Successfully queried - Tweet ID: 675006312288268288
Successfully queried - Tweet ID: 675003128568291329
Successfully queried - Tweet ID: 674999807681908736
Successfully queried - Tweet ID: 674805413498527744
Successfully queried - Tweet ID: 674800520222154752
Successfully queried - Tweet ID: 674793399141146624
Successfully queried - Tweet ID: 674790488185167872
Successfully queried - Tweet ID: 674788554665512960
Successfully queried - Tweet ID: 674781762103414784
Successfully queried - Tweet ID: 674774481756377088
Successfully queried - Tweet ID: 674767892831932416
Successfully queried - Tweet ID: 674764817387900928
Successfully queried - Tweet ID: 674754018082705410
Successfully queried - Tweet ID: 674752233200820224
Successfully queried - Tweet ID: 674743008475090944
Successfully queried - Tweet ID: 674742531037511680
Successfully queried - Tweet ID: 674739953134403584
Successfully queried - Tweet ID: 674737130913071104
Successfully queried - Tweet ID: 674690135443775488
Successfully queried - Tweet ID: 674670581682434048
Successfully queried - Tweet ID: 674664755118911488
Successfully queried - Tweet ID: 674646392044941312
Successfully queried - Tweet ID: 674644256330530816
Successfully queried - Tweet ID: 674638615994089473
Successfully queried - Tweet ID: 674632714662858753
Successfully queried - Tweet ID: 674606911342424069
Successfully queried - Tweet ID: 674468880899788800
Successfully queried - Tweet ID: 674447403907457024
Successfully queried - Tweet ID: 674436901579923456
Successfully queried - Tweet ID: 674422304705744896
Successfully queried - Tweet ID: 674416750885273600
Successfully queried - Tweet ID: 674410619106390016
Successfully queried - Tweet ID: 674394782723014656
Successfully queried - Tweet ID: 674372068062928900
Successfully queried - Tweet ID: 674330906434379776
Successfully queried - Tweet ID: 674318007229923329
Successfully queried - Tweet ID: 674307341513269249
Successfully queried - Tweet ID: 674291837063053312
Successfully queried - Tweet ID: 674271431610523648
Successfully queried - Tweet ID: 674269164442398721
Successfully queried - Tweet ID: 674265582246694913
Successfully queried - Tweet ID: 674262580978937856
Successfully queried - Tweet ID: 674255168825880576
Successfully queried - Tweet ID: 674082852460433408
Successfully queried - Tweet ID: 674075285688614912
Successfully queried - Tweet ID: 674063288070742018
Successfully queried - Tweet ID: 674053186244734976
Successfully queried - Tweet ID: 674051556661161984
Successfully queried - Tweet ID: 674045139690631169
Successfully queried - Tweet ID: 674042553264685056
Successfully queried - Tweet ID: 674038233588723717
Successfully queried - Tweet ID: 674036086168010753
Successfully queried - Tweet ID: 674024893172875264
Successfully queried - Tweet ID: 674019345211760640
Successfully queried - Tweet ID: 674014384960745472
Successfully queried - Tweet ID: 674008982932058114
Successfully queried - Tweet ID: 673956914389192708
Successfully queried - Tweet ID: 673919437611909120
Successfully queried - Tweet ID: 673906403526995968
Successfully queried - Tweet ID: 673887867907739649
Successfully queried - Tweet ID: 673716320723169284
Successfully queried - Tweet ID: 673715861853720576
Successfully queried - Tweet ID: 673711475735838725
Successfully queried - Tweet ID: 673709992831262724
Successfully queried - Tweet ID: 673708611235921920
Successfully queried - Tweet ID: 673707060090052608
Successfully queried - Tweet ID: 673705679337693185
Successfully queried - Tweet ID: 673700254269775872
Successfully queried - Tweet ID: 673697980713705472
Successfully queried - Tweet ID: 673689733134946305
Successfully queried - Tweet ID: 673688752737402881
Successfully queried - Tweet ID: 673686845050527744
Successfully queried - Tweet ID: 673680198160809984
Successfully queried - Tweet ID: 673662677122719744
Successfully queried - Tweet ID: 673656262056419329
Successfully queried - Tweet ID: 673636718965334016
Successfully queried - Tweet ID: 673612854080196609
Successfully queried - Tweet ID: 673583129559498752
Successfully queried - Tweet ID: 673580926094458881
Successfully queried - Tweet ID: 673576835670777856
Successfully queried - Tweet ID: 673363615379013632
Successfully queried - Tweet ID: 673359818736984064
Successfully queried - Tweet ID: 673355879178194945
Successfully queried - Tweet ID: 673352124999274496
Successfully queried - Tweet ID: 673350198937153538
Successfully queried - Tweet ID: 673345638550134785
Successfully queried - Tweet ID: 673343217010679808
Successfully queried - Tweet ID: 673342308415348736
Successfully queried - Tweet ID: 673320132811366400
Successfully queried - Tweet ID: 673317986296586240
Successfully queried - Tweet ID: 673295268553605120
Successfully queried - Tweet ID: 673270968295534593
Successfully queried - Tweet ID: 673240798075449344
Successfully queried - Tweet ID: 673213039743795200
Successfully queried - Tweet ID: 673148804208660480
Successfully queried - Tweet ID: 672997845381865473
Successfully queried - Tweet ID: 672995267319328768
Successfully queried - Tweet ID: 672988786805112832
Successfully queried - Tweet ID: 672984142909456390
Successfully queried - Tweet ID: 672980819271634944
Successfully queried - Tweet ID: 672975131468300288
Successfully queried - Tweet ID: 672970152493887488
Successfully queried - Tweet ID: 672968025906282496
Successfully queried - Tweet ID: 672964561327235073
Successfully queried - Tweet ID: 672902681409806336
Successfully queried - Tweet ID: 672898206762672129
Successfully queried - Tweet ID: 672884426393653248
Successfully queried - Tweet ID: 672877615439593473
Successfully queried - Tweet ID: 672834301050937345
Successfully queried - Tweet ID: 672828477930868736
Successfully queried - Tweet ID: 672640509974827008
Successfully queried - Tweet ID: 672622327801233409
Successfully queried - Tweet ID: 672614745925664768
Successfully queried - Tweet ID: 672609152938721280
Successfully queried - Tweet ID: 672604026190569472
Successfully queried - Tweet ID: 672594978741354496
Successfully queried - Tweet ID: 672591762242805761
Successfully queried - Tweet ID: 672591271085670400
Successfully queried - Tweet ID: 672538107540070400
Successfully queried - Tweet ID: 672523490734551040
Successfully queried - Tweet ID: 672488522314567680
Successfully queried - Tweet ID: 672482722825261057
Successfully queried - Tweet ID: 672481316919734272
Successfully queried - Tweet ID: 672475084225949696
Successfully queried - Tweet ID: 672466075045466113
Successfully queried - Tweet ID: 672272411274932228
Successfully queried - Tweet ID: 672267570918129665
Successfully queried - Tweet ID: 672264251789176834
Successfully queried - Tweet ID: 672256522047614977
Successfully queried - Tweet ID: 672254177670729728
Successfully queried - Tweet ID: 672248013293752320
Successfully queried - Tweet ID: 672245253877968896
Successfully queried - Tweet ID: 672239279297454080
Successfully queried - Tweet ID: 672231046314901505
Successfully queried - Tweet ID: 672222792075620352
Successfully queried - Tweet ID: 672205392827572224
Successfully queried - Tweet ID: 672169685991993344
Successfully queried - Tweet ID: 672160042234327040
Successfully queried - Tweet ID: 672139350159835138
Successfully queried - Tweet ID: 672125275208069120
Successfully queried - Tweet ID: 672095186491711488
Successfully queried - Tweet ID: 672082170312290304
Successfully queried - Tweet ID: 672068090318987265
Successfully queried - Tweet ID: 671896809300709376
Successfully queried - Tweet ID: 671891728106971137
Successfully queried - Tweet ID: 671882082306625538
Successfully queried - Tweet ID: 671879137494245376
Successfully queried - Tweet ID: 671874878652489728
Successfully queried - Tweet ID: 671866342182637568
Successfully queried - Tweet ID: 671855973984772097
Successfully queried - Tweet ID: 671789708968640512
Successfully queried - Tweet ID: 671768281401958400
Successfully queried - Tweet ID: 671763349865160704
Successfully queried - Tweet ID: 671744970634719232
Successfully queried - Tweet ID: 671743150407421952
Successfully queried - Tweet ID: 671735591348891648
Successfully queried - Tweet ID: 671729906628341761
Successfully queried - Tweet ID: 671561002136281088
Successfully queried - Tweet ID: 671550332464455680
Successfully queried - Tweet ID: 671547767500775424
Successfully queried - Tweet ID: 671544874165002241
Successfully queried - Tweet ID: 671542985629241344
Successfully queried - Tweet ID: 671538301157904385
Successfully queried - Tweet ID: 671536543010570240
Successfully queried - Tweet ID: 671533943490011136
Successfully queried - Tweet ID: 671528761649688577
Successfully queried - Tweet ID: 671520732782923777
Successfully queried - Tweet ID: 671518598289059840
Successfully queried - Tweet ID: 671511350426865664
Successfully queried - Tweet ID: 671504605491109889
Successfully queried - Tweet ID: 671497587707535361
Successfully queried - Tweet ID: 671488513339211776
Successfully queried - Tweet ID: 671486386088865792
Successfully queried - Tweet ID: 671485057807351808
Successfully queried - Tweet ID: 671390180817915904
Successfully queried - Tweet ID: 671362598324076544
Successfully queried - Tweet ID: 671357843010908160
Successfully queried - Tweet ID: 671355857343524864
Successfully queried - Tweet ID: 671347597085433856
Successfully queried - Tweet ID: 671186162933985280
Successfully queried - Tweet ID: 671182547775299584
Successfully queried - Tweet ID: 671166507850801152
Successfully queried - Tweet ID: 671163268581498880
Successfully queried - Tweet ID: 671159727754231808
Successfully queried - Tweet ID: 671154572044468225
Successfully queried - Tweet ID: 671151324042559489
Successfully queried - Tweet ID: 671147085991960577
Successfully queried - Tweet ID: 671141549288370177
Successfully queried - Tweet ID: 671138694582165504
Successfully queried - Tweet ID: 671134062904504320
Successfully queried - Tweet ID: 671122204919246848
Successfully queried - Tweet ID: 671115716440031232
Successfully queried - Tweet ID: 671109016219725825
Successfully queried - Tweet ID: 670995969505435648
Successfully queried - Tweet ID: 670842764863651840
Successfully queried - Tweet ID: 670840546554966016
Successfully queried - Tweet ID: 670838202509447168
Successfully queried - Tweet ID: 670833812859932673
Successfully queried - Tweet ID: 670832455012716544
Successfully queried - Tweet ID: 670826280409919488
Successfully queried - Tweet ID: 670823764196741120
Successfully queried - Tweet ID: 670822709593571328
Successfully queried - Tweet ID: 670815497391357952
Successfully queried - Tweet ID: 670811965569282048
Successfully queried - Tweet ID: 670807719151067136
Successfully queried - Tweet ID: 670804601705242624
Successfully queried - Tweet ID: 670803562457407488
Successfully queried - Tweet ID: 670797304698376195
Successfully queried - Tweet ID: 670792680469889025
Successfully queried - Tweet ID: 670789397210615808
Successfully queried - Tweet ID: 670786190031921152
Successfully queried - Tweet ID: 670783437142401025
Successfully queried - Tweet ID: 670782429121134593
Successfully queried - Tweet ID: 670780561024270336
Successfully queried - Tweet ID: 670778058496974848
Successfully queried - Tweet ID: 670764103623966721
Successfully queried - Tweet ID: 670755717859713024
Successfully queried - Tweet ID: 670733412878163972
Successfully queried - Tweet ID: 670727704916926465
Successfully queried - Tweet ID: 670717338665226240
Successfully queried - Tweet ID: 670704688707301377
Successfully queried - Tweet ID: 670691627984359425
Successfully queried - Tweet ID: 670679630144274432
Successfully queried - Tweet ID: 670676092097810432
Successfully queried - Tweet ID: 670668383499735048
Successfully queried - Tweet ID: 670474236058800128
Successfully queried - Tweet ID: 670468609693655041
Successfully queried - Tweet ID: 670465786746662913
Successfully queried - Tweet ID: 670452855871037440
Successfully queried - Tweet ID: 670449342516494336
Successfully queried - Tweet ID: 670444955656130560
Successfully queried - Tweet ID: 670442337873600512
Successfully queried - Tweet ID: 670435821946826752
Successfully queried - Tweet ID: 670434127938719744
Successfully queried - Tweet ID: 670433248821026816
Successfully queried - Tweet ID: 670428280563085312
Successfully queried - Tweet ID: 670427002554466305
Successfully queried - Tweet ID: 670421925039075328
Successfully queried - Tweet ID: 670420569653809152
Successfully queried - Tweet ID: 670417414769758208
Successfully queried - Tweet ID: 670411370698022913
Successfully queried - Tweet ID: 670408998013820928
Successfully queried - Tweet ID: 670403879788544000
Successfully queried - Tweet ID: 670385711116361728
Successfully queried - Tweet ID: 670374371102445568
Successfully queried - Tweet ID: 670361874861563904
Successfully queried - Tweet ID: 670338931251150849
Successfully queried - Tweet ID: 670319130621435904
Successfully queried - Tweet ID: 670303360680108032
Successfully queried - Tweet ID: 670290420111441920
Successfully queried - Tweet ID: 670093938074779648
Successfully queried - Tweet ID: 670086499208155136
Successfully queried - Tweet ID: 670079681849372674
Successfully queried - Tweet ID: 670073503555706880
Successfully queried - Tweet ID: 670069087419133954
Successfully queried - Tweet ID: 670061506722140161
Successfully queried - Tweet ID: 670055038660800512
Successfully queried - Tweet ID: 670046952931721218
Successfully queried - Tweet ID: 670040295598354432
Successfully queried - Tweet ID: 670037189829525505
Successfully queried - Tweet ID: 670003130994700288
Successfully queried - Tweet ID: 669993076832759809
Successfully queried - Tweet ID: 669972011175813120
Successfully queried - Tweet ID: 669970042633789440
Successfully queried - Tweet ID: 669942763794931712
Successfully queried - Tweet ID: 669926384437997569
Successfully queried - Tweet ID: 669923323644657664
Successfully queried - Tweet ID: 669753178989142016
Successfully queried - Tweet ID: 669749430875258880
Successfully queried - Tweet ID: 669684865554620416
Successfully queried - Tweet ID: 669683899023405056
Successfully queried - Tweet ID: 669682095984410625
Successfully queried - Tweet ID: 669680153564442624
Successfully queried - Tweet ID: 669661792646373376
Successfully queried - Tweet ID: 669625907762618368
Successfully queried - Tweet ID: 669603084620980224
Successfully queried - Tweet ID: 669597912108789760
Successfully queried - Tweet ID: 669583744538451968
Successfully queried - Tweet ID: 669573570759163904
Successfully queried - Tweet ID: 669571471778410496
Successfully queried - Tweet ID: 669567591774625800
Successfully queried - Tweet ID: 669564461267722241
Successfully queried - Tweet ID: 669393256313184256
Successfully queried - Tweet ID: 669375718304980992
Successfully queried - Tweet ID: 669371483794317312
Successfully queried - Tweet ID: 669367896104181761
Successfully queried - Tweet ID: 669363888236994561
Successfully queried - Tweet ID: 669359674819481600
Successfully queried - Tweet ID: 669354382627049472
Successfully queried - Tweet ID: 669353438988365824
Successfully queried - Tweet ID: 669351434509529089
Successfully queried - Tweet ID: 669328503091937280
Successfully queried - Tweet ID: 669327207240699904
Successfully queried - Tweet ID: 669324657376567296
Successfully queried - Tweet ID: 669216679721873412
Successfully queried - Tweet ID: 669214165781868544
Successfully queried - Tweet ID: 669203728096960512
Successfully queried - Tweet ID: 669037058363662336
Successfully queried - Tweet ID: 669015743032369152
Successfully queried - Tweet ID: 669006782128353280
Successfully queried - Tweet ID: 669000397445533696
Successfully queried - Tweet ID: 668994913074286592
Successfully queried - Tweet ID: 668992363537309700
Successfully queried - Tweet ID: 668989615043424256
Successfully queried - Tweet ID: 668988183816871936
Successfully queried - Tweet ID: 668986018524233728
Successfully queried - Tweet ID: 668981893510119424
Successfully queried - Tweet ID: 668979806671884288
Successfully queried - Tweet ID: 668975677807423489
Successfully queried - Tweet ID: 668967877119254528
Successfully queried - Tweet ID: 668960084974809088
Successfully queried - Tweet ID: 668955713004314625
Successfully queried - Tweet ID: 668932921458302977
Successfully queried - Tweet ID: 668902994700836864
Successfully queried - Tweet ID: 668892474547511297
Successfully queried - Tweet ID: 668872652652679168
Successfully queried - Tweet ID: 668852170888998912
Successfully queried - Tweet ID: 668826086256599040
Successfully queried - Tweet ID: 668815180734689280
Successfully queried - Tweet ID: 668779399630725120
Successfully queried - Tweet ID: 668655139528511488
Successfully queried - Tweet ID: 668645506898350081
Successfully queried - Tweet ID: 668643542311546881
Successfully queried - Tweet ID: 668641109086707712
Successfully queried - Tweet ID: 668636665813057536
Successfully queried - Tweet ID: 668633411083464705
Successfully queried - Tweet ID: 668631377374486528
Successfully queried - Tweet ID: 668627278264475648
Successfully queried - Tweet ID: 668625577880875008
Successfully queried - Tweet ID: 668623201287675904
Successfully queried - Tweet ID: 668620235289837568
Successfully queried - Tweet ID: 668614819948453888
Successfully queried - Tweet ID: 668587383441514497
Successfully queried - Tweet ID: 668567822092664832
Successfully queried - Tweet ID: 668544745690562560
Successfully queried - Tweet ID: 668542336805281792
Successfully queried - Tweet ID: 668537837512433665
Successfully queried - Tweet ID: 668528771708952576
Successfully queried - Tweet ID: 668507509523615744
Successfully queried - Tweet ID: 668496999348633600
Successfully queried - Tweet ID: 668484198282485761
Successfully queried - Tweet ID: 668480044826800133
Successfully queried - Tweet ID: 668466899341221888
Successfully queried - Tweet ID: 668297328638447616
Successfully queried - Tweet ID: 668291999406125056
Successfully queried - Tweet ID: 668286279830867968
Successfully queried - Tweet ID: 668274247790391296
Successfully queried - Tweet ID: 668268907921326080
Successfully queried - Tweet ID: 668256321989451776
Successfully queried - Tweet ID: 668248472370458624
Successfully queried - Tweet ID: 668237644992782336
Successfully queried - Tweet ID: 668226093875376128
Successfully queried - Tweet ID: 668221241640230912
Successfully queried - Tweet ID: 668204964695683073
Successfully queried - Tweet ID: 668190681446379520
Successfully queried - Tweet ID: 668171859951755264
Successfully queried - Tweet ID: 668154635664932864
Successfully queried - Tweet ID: 668142349051129856
Successfully queried - Tweet ID: 668113020489474048
Successfully queried - Tweet ID: 667937095915278337
Successfully queried - Tweet ID: 667924896115245057
Successfully queried - Tweet ID: 667915453470232577
Successfully queried - Tweet ID: 667911425562669056
Successfully queried - Tweet ID: 667902449697558528
Successfully queried - Tweet ID: 667886921285246976
Successfully queried - Tweet ID: 667885044254572545
Successfully queried - Tweet ID: 667878741721415682
Successfully queried - Tweet ID: 667873844930215936
Successfully queried - Tweet ID: 667866724293877760
Successfully queried - Tweet ID: 667861340749471744
Successfully queried - Tweet ID: 667832474953625600
Successfully queried - Tweet ID: 667806454573760512
Successfully queried - Tweet ID: 667801013445750784
Successfully queried - Tweet ID: 667793409583771648
Successfully queried - Tweet ID: 667782464991965184
Successfully queried - Tweet ID: 667773195014021121
Successfully queried - Tweet ID: 667766675769573376
Successfully queried - Tweet ID: 667728196545200128
Successfully queried - Tweet ID: 667724302356258817
Successfully queried - Tweet ID: 667550904950915073
Successfully queried - Tweet ID: 667550882905632768
Successfully queried - Tweet ID: 667549055577362432
Successfully queried - Tweet ID: 667546741521195010
Successfully queried - Tweet ID: 667544320556335104
Successfully queried - Tweet ID: 667538891197542400
Successfully queried - Tweet ID: 667534815156183040
Successfully queried - Tweet ID: 667530908589760512
Successfully queried - Tweet ID: 667524857454854144
Successfully queried - Tweet ID: 667517642048163840
Successfully queried - Tweet ID: 667509364010450944
Successfully queried - Tweet ID: 667502640335572993
Successfully queried - Tweet ID: 667495797102141441
Successfully queried - Tweet ID: 667491009379606528
Successfully queried - Tweet ID: 667470559035432960
Successfully queried - Tweet ID: 667455448082227200
Successfully queried - Tweet ID: 667453023279554560
Successfully queried - Tweet ID: 667443425659232256
Successfully queried - Tweet ID: 667437278097252352
Successfully queried - Tweet ID: 667435689202614272
Successfully queried - Tweet ID: 667405339315146752
Successfully queried - Tweet ID: 667393430834667520
Successfully queried - Tweet ID: 667369227918143488
Successfully queried - Tweet ID: 667211855547486208
Successfully queried - Tweet ID: 667200525029539841
Successfully queried - Tweet ID: 667192066997374976
Successfully queried - Tweet ID: 667188689915760640
Successfully queried - Tweet ID: 667182792070062081
Successfully queried - Tweet ID: 667177989038297088
Successfully queried - Tweet ID: 667176164155375616
Successfully queried - Tweet ID: 667174963120574464
Successfully queried - Tweet ID: 667171260800061440
Successfully queried - Tweet ID: 667165590075940865
Successfully queried - Tweet ID: 667160273090932737
Successfully queried - Tweet ID: 667152164079423490
Successfully queried - Tweet ID: 667138269671505920
Successfully queried - Tweet ID: 667119796878725120
Successfully queried - Tweet ID: 667090893657276420
Successfully queried - Tweet ID: 667073648344346624
Successfully queried - Tweet ID: 667070482143944705
Successfully queried - Tweet ID: 667065535570550784
Successfully queried - Tweet ID: 667062181243039745
Successfully queried - Tweet ID: 667044094246576128
Successfully queried - Tweet ID: 667012601033924608
Successfully queried - Tweet ID: 666996132027977728
Successfully queried - Tweet ID: 666983947667116034
Successfully queried - Tweet ID: 666837028449972224
Successfully queried - Tweet ID: 666835007768551424
Successfully queried - Tweet ID: 666826780179869698
Successfully queried - Tweet ID: 666817836334096384
Successfully queried - Tweet ID: 666804364988780544
Successfully queried - Tweet ID: 666786068205871104
Successfully queried - Tweet ID: 666781792255496192
Successfully queried - Tweet ID: 666776908487630848
Successfully queried - Tweet ID: 666739327293083650
Successfully queried - Tweet ID: 666701168228331520
Successfully queried - Tweet ID: 666691418707132416
Successfully queried - Tweet ID: 666649482315059201
Successfully queried - Tweet ID: 666644823164719104
Successfully queried - Tweet ID: 666454714377183233
Successfully queried - Tweet ID: 666447344410484738
Successfully queried - Tweet ID: 666437273139982337
Successfully queried - Tweet ID: 666435652385423360
Successfully queried - Tweet ID: 666430724426358785
Successfully queried - Tweet ID: 666428276349472768
Successfully queried - Tweet ID: 666421158376562688
Successfully queried - Tweet ID: 666418789513326592
Successfully queried - Tweet ID: 666411507551481857
Successfully queried - Tweet ID: 666407126856765440
Successfully queried - Tweet ID: 666396247373291520
Successfully queried - Tweet ID: 666373753744588802
Successfully queried - Tweet ID: 666362758909284353
Successfully queried - Tweet ID: 666353288456101888
Successfully queried - Tweet ID: 666345417576210432
Successfully queried - Tweet ID: 666337882303524864
Successfully queried - Tweet ID: 666293911632134144
Successfully queried - Tweet ID: 666287406224695296
Successfully queried - Tweet ID: 666273097616637952
Successfully queried - Tweet ID: 666268910803644416
Successfully queried - Tweet ID: 666104133288665088
Successfully queried - Tweet ID: 666102155909144576
Successfully queried - Tweet ID: 666099513787052032
Successfully queried - Tweet ID: 666094000022159362
Successfully queried - Tweet ID: 666082916733198337
Successfully queried - Tweet ID: 666073100786774016
Successfully queried - Tweet ID: 666071193221509120
Successfully queried - Tweet ID: 666063827256086533
Successfully queried - Tweet ID: 666058600524156928
Successfully queried - Tweet ID: 666057090499244032
Successfully queried - Tweet ID: 666055525042405380
Successfully queried - Tweet ID: 666051853826850816
Successfully queried - Tweet ID: 666050758794694657
Successfully queried - Tweet ID: 666049248165822465
Successfully queried - Tweet ID: 666044226329800704
Successfully queried - Tweet ID: 666033412701032449
Successfully queried - Tweet ID: 666029285002620928
Successfully queried - Tweet ID: 666020888022790149

In [61]:

# Get the duration for the whole data collection from twitter
print("Twitter Query and file writing duration: " + str(stop - start))

Twitter Query and file writing duration: 1430.7260451979819

Now as tweet_json.txt file has been properly created, we can read it and create the associated pandas DataFrame with at a minimum "tweet ID", "retweet count", and "favorite count" as columns.

In [791]:

# A dictionary to build and convert to a DataFrame later
df_tweet_json = []

We've got a basic understanding of the tweet json object from here : https://developer.twitter.com/en/docs/tweets/data-dictionary/overview/tweet-object.html. So I choose to collect "created_at", "id", "retweet_count", "favorite_count" and "full_text".

In [792]:

# Load "tweet_json.txt"
with open('tweet_json.txt') as tweet_json:
    
    # Get each line and load it as a json object
    for a_line in tweet_json:
        line_data_json = json.loads(a_line)
        
        # Get the tweet ID
        tweet_id = line_data_json['id']
        
        # Get the tweet retweet count
        retweet_count = line_data_json['retweet_count']
        
        # Get the tweet favorites count
        favorite_count = line_data_json['favorite_count']
        
        # Get the tweet creation date
        creation_date = line_data_json['created_at']
        
        # Get the tweet full text
        tweet_full_text = line_data_json['full_text']
        
        
        # Add the extracted data to the dictionary
        df_tweet_json.append({'tweet_id': tweet_id,
                            'created_at': creation_date,
                            'retweet_count': retweet_count,
                            'favorite_count': favorite_count,
                            'tweet_full_text': tweet_full_text})

In [793]:

# Convert the dictionary to a DataFrame
df_twitter = pd.DataFrame(df_tweet_json, columns = ['tweet_id', 'created_at', 'retweet_count', 'favorite_count', 'tweet_full_text'])

In [794]:

# Create a copy to use for assessing and cleaning
df_twitter_clean = df_twitter.copy()

In [795]:

# Get a view on the dataframe
df_twitter_clean.head()

Out[795]:

	tweet_id	created_at	retweet_count	favorite_count	tweet_full_text
0	892420643555336193	Tue Aug 01 16:23:56 +0000 2017	8204	37636	This is Phineas. He's a mystical boy. Only ever appears in the hole of a donut. 13/10 https://t.co/MgUWQ76dJU
1	892177421306343426	Tue Aug 01 00:17:27 +0000 2017	6071	32337	This is Tilly. She's just checking pup on you. Hopes you're doing ok. If not, she's available for pats, snugs, boops, the whole bit. 13/10 https://t.co/0Xxu71qeIV
2	891815181378084864	Mon Jul 31 00:18:03 +0000 2017	4011	24363	This is Archie. He is a rare Norwegian Pouncing Corgo. Lives in the tall grass. You never know when one may strike. 12/10 https://t.co/wUnZnhtVJB
3	891689557279858688	Sun Jul 30 15:58:51 +0000 2017	8366	40959	This is Darla. She commenced a snooze mid meal. 13/10 happens to the best of us https://t.co/tD36da7qLQ
4	891327558926688256	Sat Jul 29 16:00:24 +0000 2017	9063	39161	This is Franklin. He would like you to stop calling him "cute." He is a very fierce shark and should be respected as such. 12/10 #BarkWeek https://t.co/AtUZn91f7f

At this point, We've got three DataFrames for the next assessment and cleaning activities:

df_archive_clean : WeRateDogs twitter enhanced archive. We know that the ratings are probably not all correct, same for the dog names and dog stages.
df_image_clean : the tweets images prediction, from image-predictions.tsv which we gathered from an URL.
df_twitter_clean: additional information - retweet counts, favorite counts - gathered from twitter because missing from df_archive_clean

Assessing¶

We focus first on missing data, completness, format as quality issues. Then we examine tidiness.

1. The WeRateDogs twitter archive¶

Here are the information we already know about df_archive_clean : the ratings are probably not all correct, same for the dog names and dog stages. So we'll need to assess those columns.

In [796]:

# Get a view on the dataset
df_archive_clean.sample(5)

Out[796]:

	tweet_id	in_reply_to_status_id	in_reply_to_user_id	timestamp	source	text	retweeted_status_id	retweeted_status_user_id	retweeted_status_timestamp	expanded_urls	rating_numerator	rating_denominator	name	doggo	floofer	pupper	puppo
2147	669753178989142016	NaN	NaN	2015-11-26 05:42:55 +0000	<a href="http://twitter.com/download/iphone" rel="nofollow">Twitter for iPhone</a>	Meet Chester. He just ate a lot and now he can't move. 10/10 that's going to be me in about 17 hours https://t.co/63jh1tYZa5	NaN	NaN	NaN	https://twitter.com/dog_rates/status/669753178989142016/photo/1	10	10	Chester	None	None	None	None
603	798628517273620480	NaN	NaN	2016-11-15 20:47:30 +0000	<a href="http://twitter.com/download/iphone" rel="nofollow">Twitter for iPhone</a>	RT @dog_rates: This a Norwegian Pewterschmidt named Tickles. Ears for days. 12/10 I care deeply for Tickles https://t.co/0aDF62KVP7	6.675094e+17	4.196984e+09	2015-11-20 01:06:48 +0000	https://twitter.com/dog_rates/status/667509364010450944/photo/1,https://twitter.com/dog_rates/status/667509364010450944/photo/1	12	10	None	None	None	None	None
2308	666817836334096384	NaN	NaN	2015-11-18 03:18:55 +0000	<a href="http://twitter.com/download/iphone" rel="nofollow">Twitter for iPhone</a>	This is Jeph. He is a German Boston Shuttlecock. Enjoys couch. Lost body during French Revolution. True hero 9/10 https://t.co/8whlkYw3mO	NaN	NaN	NaN	https://twitter.com/dog_rates/status/666817836334096384/photo/1	9	10	Jeph	None	None	None	None
468	817056546584727552	NaN	NaN	2017-01-05 17:13:55 +0000	<a href="http://twitter.com/download/iphone" rel="nofollow">Twitter for iPhone</a>	This is Chloe. She fell asleep at the wheel. Absolute menace on the roadways. Sneaky tongue slip tho. 11/10 https://t.co/r6SLVN2VUH	NaN	NaN	NaN	https://twitter.com/dog_rates/status/817056546584727552/photo/1	11	10	Chloe	None	None	None	None
726	782598640137187329	NaN	NaN	2016-10-02 15:10:30 +0000	<a href="http://twitter.com/download/iphone" rel="nofollow">Twitter for iPhone</a>	This is Timmy. He's quite large. According to a trusted source it's actually a dog wearing a dog suit. 11/10 https://t.co/BIUchFwHqn	NaN	NaN	NaN	https://twitter.com/dog_rates/status/782598640137187329/photo/1	11	10	Timmy	None	None	None	None

Elements from a first visual observation:

"source" column contains html tags. We do not consider this as an issue regarding the analysis we plan to conduct.
"expanded_urls" contains some duplicated values.

In [797]:

# Get the dataset size
df_archive_clean.shape

Out[797]:

(2356, 17)

In [798]:

# Do we have duplicates ?
sum(df_archive_clean.duplicated())

Out[798]:

In [799]:

# What about missing data ?
df_archive_clean.isnull().sum()

Out[799]:

tweet_id                      0   
in_reply_to_status_id         2278
in_reply_to_user_id           2278
timestamp                     0   
source                        0   
text                          0   
retweeted_status_id           2175
retweeted_status_user_id      2175
retweeted_status_timestamp    2175
expanded_urls                 59  
rating_numerator              0   
rating_denominator            0   
name                          0   
doggo                         0   
floofer                       0   
pupper                        0   
puppo                         0   
dtype: int64

Basically, the rows with data in the "retweeted_status" columns are the ones related to a retweet. As not having retweet is a prerequisite, I will delete all the rows with values in the "retweeted_status*" columns. I will do the same for the "reply_to" columns. Following to that, I will delete those columns because those will not be useful anymore.

Quality Issue 1 : The WeRateDgos twitter archive enhanced contains retweets, which could not be part or our study¶

We observe 59 missing expanded_urls.

Quality Issue 2 : The WeRateDgos twitter archive enhanced has 59 missing value from the "expanded_urls" column¶

Now, let's check the unique values for some columns, the ones we are most likely to use further.

In [800]:

# Get the unique values for rating numerator, as we knew we might have issues there
df_archive_clean.rating_numerator.unique()

Out[800]:

array([  13,   12,   14,    5,   17,   11,   10,  420,  666,    6,   15,
        182,  960,    0,   75,    7,   84,    9,   24,    8,    1,   27,
          3,    4,  165, 1776,  204,   50,   99,   80,   45,   60,   44,
        143,  121,   20,   26,    2,  144,   88])

In [801]:

# is the 0 value a normal situation ?
df_archive_clean.query('rating_numerator == 0')[['rating_numerator', 'rating_denominator', 'text']]

Out[801]:

	rating_numerator	rating_denominator	text
315	0	10	When you're so blinded by your systematic plagiarism that you forget what day it is. 0/10 https://t.co/YbEJPkg4Ag
1016	0	10	PUPDATE: can't see any. Even if I could, I couldn't reach them to pet. 0/10 much disappointment https://t.co/c7WXaB2nqX

We've got some very high values ! let's check the context to see if this situation really makes sense.

In [802]:

# Set the column width to max, so we can easily view all the cells text data
pd.set_option('display.max_colwidth', -1)

In [803]:

# Get the cases with a suspicious denominator so we can observe them visually
df_denominator_issues = df_archive_clean.query('rating_denominator != 10')[['rating_numerator', 'rating_denominator', 'text']]
df_denominator_issues

Out[803]:

	rating_numerator	rating_denominator	text
313	960	0	@jonnysun @Lin_Manuel ok jomny I know you're excited but 960/00 isn't a valid rating, 13/10 is tho
342	11	15	@docmisterio account started on 11/15/15
433	84	70	The floofs have been released I repeat the floofs have been released. 84/70 https://t.co/NIYC820tmd
516	24	7	Meet Sam. She smiles 24/7 & secretly aspires to be a reindeer. \nKeep Sam smiling by clicking and sharing this link:\nhttps://t.co/98tB8y7y7t https://t.co/LouL5vdvxx
784	9	11	RT @dog_rates: After so many requests, this is Bretagne. She was the last surviving 9/11 search dog, and our second ever 14/10. RIP https:/…
902	165	150	Why does this never happen at my front door... 165/150 https://t.co/HmwrdfEfUE
1068	9	11	After so many requests, this is Bretagne. She was the last surviving 9/11 search dog, and our second ever 14/10. RIP https://t.co/XAVDNDaVgQ
1120	204	170	Say hello to this unbelievably well behaved squad of doggos. 204/170 would try to pet all at once https://t.co/yGQI3He3xv
1165	4	20	Happy 4/20 from the squad! 13/10 for all https://t.co/eV1diwds8a
1202	50	50	This is Bluebert. He just saw that both #FinalFur match ups are split 50/50. Amazed af. 11/10 https://t.co/Kky1DPG4iq
1228	99	90	Happy Saturday here's 9 puppers on a bench. 99/90 good work everybody https://t.co/mpvaVxKmc1
1254	80	80	Here's a brigade of puppers. All look very prepared for whatever happens next. 80/80 https://t.co/0eb7R1Om12
1274	45	50	From left to right:\nCletus, Jerome, Alejandro, Burp, & Titson\nNone know where camera is. 45/50 would hug all at once https://t.co/sedre1ivTK
1351	60	50	Here is a whole flock of puppers. 60/50 I'll take the lot https://t.co/9dpcw6MdWa
1433	44	40	Happy Wednesday here's a bucket of pups. 44/40 would pet all at once https://t.co/HppvrYuamZ
1598	4	20	Yes I do realize a rating of 4/20 would've been fitting. However, it would be unjust to give these cooperative pups that low of a rating
1634	143	130	Two sneaky puppers were not initially seen, moving the rating to 143/130. Please forgive us. Thank you https://t.co/kRK51Y5ac3
1635	121	110	Someone help the girl is being mugged. Several are distracting her while two steal her shoes. Clever puppers 121/110 https://t.co/1zfnTJLt55
1662	7	11	This is Darrel. He just robbed a 7/11 and is in a high speed police chase. Was just spotted by the helicopter 10/10 https://t.co/7EsP8LmSp5
1663	20	16	I'm aware that I could've said 20/16, but here at WeRateDogs we are very professional. An inconsistent rating scale is simply irresponsible
1779	144	120	IT'S PUPPERGEDDON. Total of 144/120 ...I think https://t.co/ZanVtAtvIq
1843	88	80	Here we have an entire platoon of puppers. Total score: 88/80 would pet all at once https://t.co/y93p6FLvVw
2335	1	2	This is an Albanian 3 1/2 legged Episcopalian. Loves well-polished hardwood flooring. Penis on the collar. 9/10 https://t.co/d9NcXFKwLv

We observe :

One case where the extracted information - 24/7 - is not a rating
Normal situations, as previously observed : the denominator is high but inline with the numerator value
Other situations where the first "rating like" values encountered have been extracted instead of the second ones. For example we see a line where 50/50 have been extracted instead of 11/10, which is the real rating within the same text message.

Quality Issue 4 : WeRateDgos twitter archive - in some cases, the relevant rating information, consequently rating_numerator and rating_denominator, have not been extracted.¶

We continue to check unique values for some columns.
Visually, we saw "none" as dog names. Let's dig in that.

In [804]:

# Check the dog names, as we might have issues there. We get 20 rows for a visual observation
df_name_issues = df_archive_clean.query('name == "None"')[['name', 'text']]
df_name_issues.sample(20)

Out[804]:

	name	text
1928	None	Herd of wild dogs here. Not sure what they're trying to do. No real goals in life. 3/10 find your purpose puppers https://t.co/t5ih0VrK02
2013	None	Exotic underwater dog here. Very shy. Wont return tennis balls I toss him. Never been petted. 5/10 I bet he's soft https://t.co/WH7Nzc5IBA
1783	None	Endangered triangular pup here. Could be a wizard. Caught mid-laugh. No legs. Just fluff. Probably a wizard. 9/10 https://t.co/GFVIHIod0Z
2067	None	Neat pup here. Enjoys lettuce. Long af ears. Short lil legs. Hops surprisingly high for dog. 9/10 still very petable https://t.co/HYR611wiA4
494	None	We only rate dogs. Please don't send in other things like this very good Christmas tree. Thank you... 13/10 https://t.co/rvSANEsQZJ
1408	None	ERMAHGERD 12/10 please enjoy https://t.co/7WrAWKdBac
212	None	RT @eddie_coe98: Thanks @dog_rates completed my laptop. 10/10 would buy again https://t.co/bO0rThDlXI
179	None	@Marc_IRL pixelated af 12/10
2131	None	"Hi yes this is dog. I can't help with that s- sir please... the manager isn't in right n- well that was rude"\n10/10 https://t.co/DuQXATW27f
1935	None	This pup is sad bc he didn't get to be the toy car. Also he has shitty money management skills. 10/10 still cute tho https://t.co/PiSXXZjDSJ
268	None	Here we have some incredible doggos for #K9VeteransDay. All brave as h*ck. Salute your dog in solidarity. 14/10 for all https://t.co/SVNMdFqKDL
1901	None	Two gorgeous dogs here. Little waddling dog is a rebel. Refuses to look at camera. Must be a preteen. 5/10 & 8/10 https://t.co/YPfw7oahbD
189	None	@s8n You tried very hard to portray this good boy as not so good, but you have ultimately failed. His goodness shines through. 666/10
1068	None	After so many requests, this is Bretagne. She was the last surviving 9/11 search dog, and our second ever 14/10. RIP https://t.co/XAVDNDaVgQ
493	None	Here's a doggo who has concluded that Christmas is entirely too bright. Requests you tone it down a notch. 11/10 https://t.co/cD967DjnIn
2089	None	Two obedient dogs here. Left one has extra leg sticking out of its back. They each get 9/10. Would pet both at once https://t.co/RGcNPsmAfY
1532	None	"I'm the only one that ever does anything in this household" 10/10 https://t.co/V8HcVIh4jt
1248	None	"Please, no puparazzi" 11/10 https://t.co/nJIXSPfedK
1234	None	Please don't send in any more polar bears. We only rate dogs. Thank you... 10/10 https://t.co/83RGhdIQz2
568	None	RT @ChinoChinako: They're good products, Brent\n\nMug holds drinks; hoodie is comfy af. 13/10 \n\nPuppy Aika h*cking agrees. @dog_rates https:/…

We observe that sometimes we have names in the text, so we still have the possibility to extract them. For example, at index 72, we could have "Martha", at index 2237, we have "Oliver". The "None" here as a string could be confusing. We do not consider this as an issue. Basically we just want to be able to distinguish when we do have a name, from when we don't.

Quality Issue 5 : WeRateDgos twitter archive - We have "None" as name even if, sometimes, there is an available name in the text. So few names are missing and could be retrieved.¶

We observe "None" values in the dog stages "doggo, floofer, pupper, puppo" columns.

In [805]:

# Get all the rows where the dog stage has not been identified, meaning set to "None"
df_dogtionary_issue = df_archive_clean.query('doggo == "None" and floofer == "None" and pupper == "None" and puppo == "None"')[['text', 'doggo', 'floofer', 'pupper', 'puppo']]
df_dogtionary_issue.shape

Out[805]:

(1976, 5)

In [806]:

# Do we have any puppo in the associated text content ? meaning not identified ?
df_dogtionary_issue['text'].apply(lambda x: ('puppo' in x)).value_counts()

Out[806]:

False    1969
True     7   
Name: text, dtype: int64

In [807]:

# Do we have any doggo in the associated text content ? meaning not identified ?
df_dogtionary_issue['text'].apply(lambda x: ('doggo' in x)).value_counts()

Out[807]:

False    1966
True     10  
Name: text, dtype: int64

In [808]:

# Do we have any floofer in the associated text content ? meaning not identified ?
df_dogtionary_issue['text'].apply(lambda x: ('floofer' in x)).value_counts()

Out[808]:

False    1976
Name: text, dtype: int64

In [809]:

# Do we have any pupper in the associated text content ? meaning not identified ?
df_dogtionary_issue['text'].apply(lambda x: ('pupper' in x)).value_counts()

Out[809]:

False    1952
True     24  
Name: text, dtype: int64

As a conclusion, we have a quality issue. Clearly we have the possibility to have several dog stages within the same text. So the stage extrated might not be the right one, sometimes. We do not consider this as an issue, because of the potential low number of rows impacted.

Quality Issue 6 : WeRateDgos twitter archive - We have missing dog stages, meaning stages not properly extracted. This is the case for "puppo", "doggo" and "pupper".¶

In [810]:

# Check the duplicated expanded_urls we observed previously
df_archive_clean['expanded_urls'].str.contains(',').value_counts()

Out[810]:

False    1658
True     639 
Name: expanded_urls, dtype: int64

Quality Issue 7: WeRateDgos twitter archive - "expanded_urls" column contains duplicated urls¶

Now we check the column types.

In [811]:

# Examine the columns type
df_archive_clean.dtypes

Out[811]:

tweet_id                      int64  
in_reply_to_status_id         float64
in_reply_to_user_id           float64
timestamp                     object 
source                        object 
text                          object 
retweeted_status_id           float64
retweeted_status_user_id      float64
retweeted_status_timestamp    object 
expanded_urls                 object 
rating_numerator              int64  
rating_denominator            int64  
name                          object 
doggo                         object 
floofer                       object 
pupper                        object 
puppo                         object 
dtype: object

From the previous investigations, we already know that "source", "text", "name" and dog stages are all string typed.

In [812]:

# So we get more detailed visibility on timestamp. 
type(df_archive_clean.timestamp[0])

Out[812]:

str

Quality Issue 8: WeRateDgos twitter archive - timestamp is using string type.¶

From observation: the dog stages have been spread as differents columns. This is a tidiness issue.

Tidiness Issue 1 : WeRateDgos twitter archive - Dog stages have been spread as columns.¶

2. The tweet image predictions¶

In [813]:

# Get a view on the dataframe
df_image_clean.head()

Out[813]:

	tweet_id	jpg_url	img_num	p1	p1_conf	p1_dog	p2	p2_conf	p2_dog	p3	p3_conf	p3_dog
0	666020888022790149	https://pbs.twimg.com/media/CT4udn0WwAA0aMy.jpg	1	Welsh_springer_spaniel	0.465074	True	collie	0.156665	True	Shetland_sheepdog	0.061428	True
1	666029285002620928	https://pbs.twimg.com/media/CT42GRgUYAA5iDo.jpg	1	redbone	0.506826	True	miniature_pinscher	0.074192	True	Rhodesian_ridgeback	0.072010	True
2	666033412701032449	https://pbs.twimg.com/media/CT4521TWwAEvMyu.jpg	1	German_shepherd	0.596461	True	malinois	0.138584	True	bloodhound	0.116197	True
3	666044226329800704	https://pbs.twimg.com/media/CT5Dr8HUEAA-lEu.jpg	1	Rhodesian_ridgeback	0.408143	True	redbone	0.360687	True	miniature_pinscher	0.222752	True
4	666049248165822465	https://pbs.twimg.com/media/CT5IQmsXIAAKY4A.jpg	1	miniature_pinscher	0.560311	True	Rottweiler	0.243682	True	Doberman	0.154629	True

Observations: we have several predictions and their confidences columns. A column to inform if the prediction is a dog or not is not necessary assuming that we only need dog's prediction here.

Quality issue 11 : Tweets images predictions - predictions and the associated confidences are spread over several columns.¶

Observations: we have tweets informations here also. So tweets informations are spread across several files. This is a tidiness issue.

Tidiness Issue 2 : Tweets information spreaded over several files and dataframes.¶

In [814]:

# Get the dataset size
df_image_clean.shape

Out[814]:

(2075, 12)

Smaller than the twitter archive dataset.

In [815]:

# Check the duplicated rows
sum(df_image_clean.duplicated())

Out[815]:

In [816]:

# Do we have missing data somewhere ?
df_image_clean.isnull().sum()

Out[816]:

tweet_id    0
jpg_url     0
img_num     0
p1          0
p1_conf     0
p1_dog      0
p2          0
p2_conf     0
p2_dog      0
p3          0
p3_conf     0
p3_dog      0
dtype: int64

In [817]:

# Get a view on the types
df_image_clean.dtypes

Out[817]:

tweet_id    int64  
jpg_url     object 
img_num     int64  
p1          object 
p1_conf     float64
p1_dog      bool   
p2          object 
p2_conf     float64
p2_dog      bool   
p3          object 
p3_conf     float64
p3_dog      bool   
dtype: object

In [818]:

# Get the distinct values for p1_dog, p2_dog and p3_dog
df_image_clean.p1_dog.value_counts(), df_image_clean.p2_dog.value_counts(), df_image_clean.p2_dog.value_counts()

Out[818]:

(True     1532
 False    543 
 Name: p1_dog, dtype: int64, True     1553
 False    522 
 Name: p2_dog, dtype: int64, True     1553
 False    522 
 Name: p2_dog, dtype: int64)

Quality Issue 9 : Tweets images predictions - We have rows which are not dogs images predictions¶

3. Data retrieved via twitter API¶

In [819]:

# Get a view in the information gathered vi twitter API
df_twitter_clean.head()

Out[819]:

	tweet_id	created_at	retweet_count	favorite_count	tweet_full_text
0	892420643555336193	Tue Aug 01 16:23:56 +0000 2017	8204	37636	This is Phineas. He's a mystical boy. Only ever appears in the hole of a donut. 13/10 https://t.co/MgUWQ76dJU
1	892177421306343426	Tue Aug 01 00:17:27 +0000 2017	6071	32337	This is Tilly. She's just checking pup on you. Hopes you're doing ok. If not, she's available for pats, snugs, boops, the whole bit. 13/10 https://t.co/0Xxu71qeIV
2	891815181378084864	Mon Jul 31 00:18:03 +0000 2017	4011	24363	This is Archie. He is a rare Norwegian Pouncing Corgo. Lives in the tall grass. You never know when one may strike. 12/10 https://t.co/wUnZnhtVJB
3	891689557279858688	Sun Jul 30 15:58:51 +0000 2017	8366	40959	This is Darla. She commenced a snooze mid meal. 13/10 happens to the best of us https://t.co/tD36da7qLQ
4	891327558926688256	Sat Jul 29 16:00:24 +0000 2017	9063	39161	This is Franklin. He would like you to stop calling him "cute." He is a very fierce shark and should be respected as such. 12/10 #BarkWeek https://t.co/AtUZn91f7f

In [820]:

# Get the size
df_twitter_clean.shape

Out[820]:

(2337, 5)

It is 19 rows smaller than df_archive_clean which is the WeRateDogs twitter archive.

In [821]:

# Check the duplicated rows
sum(df_twitter_clean.duplicated())

Out[821]:

In [822]:

# Do we have missing data somewhere ? - normally no, considering the way we collected the data
df_twitter_clean.isnull().sum()

Out[822]:

tweet_id           0
created_at         0
retweet_count      0
favorite_count     0
tweet_full_text    0
dtype: int64

In [823]:

# Get a view on the types
df_twitter_clean.dtypes

Out[823]:

tweet_id           int64 
created_at         object
retweet_count      int64 
favorite_count     int64 
tweet_full_text    object
dtype: object

In [824]:

# Inspect the type for created_at which should be timestamp
type(df_twitter_clean.created_at[0])

Out[824]:

str

Quality Issue 10 : Data retrieved via twitter API - "created_at" column type is not timestamp¶

4. Synthesis¶

Tidiness issues identified :¶

WeRateDgos twitter archive - Dog stages have been spread as columns.
Tweets information spreaded over several files and dataframes.

Quality issues identified :¶

The WeRateDgos twitter archive enhanced contains retweets, which could not be part or our study
The WeRateDgos twitter archive enhanced has 59 missing value from the "expanded_urls" column
REMOVED!
WeRateDgos twitter archive - in some cases, the relevant rating information, consequently rating_numerator and rating_denominator, have not been extracted.
WeRateDgos twitter archive - We have "None" as name even if, sometimes, there is an available name in the text. So few names are missing and could be retrieved.
WeRateDgos twitter archive - We have missing dog stages, meaning stages not properly extracted. This is the case for "puppo", "doggo" and "pupper".
WeRateDgos twitter archive - "expanded_urls" column contains duplicated urls.
WeRateDgos twitter archive - timestamp is using string type.
Tweets images predictions - We have rows which are not dogs images predictions.
Data retrieved via twitter API - "created_at" column type is not timestamp.
Tweets images predictions - predictions and the associated confidences are spread over several columns.

Cleaning¶

Approach¶

First, we address the missing data and completness issues (Quality issues 1, 2, 4, 5, 6, 9)
Then we resolve the tidiness problems we identified.
Finally we will correct the quality issues (Quality issues 7, 8, 10)

Cleaning - Quality Issue 1 : The WeRateDgos twitter archive enhanced contains retweets, which could not be part or our study¶

We found out previously that we have lines with values in retweeted_status_id, retweeted_status_user_id and retweeted_status_timestamp columns. We checked their meaning from the doc here : https://developer.twitter.com/en/docs/tweets/data-dictionary/overview/tweet-object.html
The requirements we have are regarding retweets only. For simplification, we make the choice to remove the replys also.

Define : we remove the rows with existing values in "retweeted_status_id" and "in_reply_to_status_id"

In [825]:

# Code : It is more easy to keep the rows with NaN. 
# Use value != value as an isnan check, 
# my source (from internet search) : https://stackoverflow.com/questions/26535563/querying-for-nan-and-other-names-in-pandas
df_archive_clean = df_archive_clean.query('retweeted_status_id != retweeted_status_id and in_reply_to_status_id != in_reply_to_status_id')

In [826]:

# Test : the number of null values is the number of rows
assert df_archive_clean.shape[0] == df_archive_clean.retweeted_status_id.isnull().sum()
assert df_archive_clean.shape[0] == df_archive_clean.in_reply_to_status_id.isnull().sum()

In [827]:

# Closure : the following columns are not needed anymore : in_reply_to_status_id, in_reply_to_user_id
# retweeted_status_id, retweeted_status_user_id and retweeted_status_timestamp
df_archive_clean.drop('in_reply_to_status_id', axis=1, inplace=True)
df_archive_clean.drop('in_reply_to_user_id', axis=1, inplace=True)
df_archive_clean.drop('retweeted_status_id', axis=1, inplace=True)
df_archive_clean.drop('retweeted_status_user_id', axis=1, inplace=True)
df_archive_clean.drop('retweeted_status_timestamp', axis=1, inplace=True)

Cleaning - Quality Issue 2 : The WeRateDgos twitter archive enhanced has 59 missing value from the "expanded_urls" column¶

In [828]:

# We still have missing value
df_archive_clean.expanded_urls.isnull().sum()

Out[828]:

In [829]:

# Define and Code : we drop the rows with missing values
df_archive_clean.dropna(inplace=True)

In [830]:

# Test : do we still have missing values ?
df_archive_clean.isnull().sum().any()

Out[830]:

False

Cleaning - Quality Issue 4 : WeRateDgos twitter archive - in some cases, the relevant rating information, consequently rating_numerator and rating_denominator, have not been extracted.¶

In [831]:

# Define : for all the 'text' cells, we find all the ratings and always take the last one. 
# Then we split in 2 parts to get the numerator and the denominator. 

# for the findall syntax: https://docs.python.org/3/library/re.html#re.findall
# for the regex, we use https://regex101.com/ with few examples

# Code : Extract numerator rating or return nothing, meaning we do not change the current rating
# The new regular expression has been provided by the reviewer
def extract_numerator_rating(row):
    match = re.findall('((?:\d+\.)?\d+)\/(\d+)', row['text'])
    if match:
        return float(match[-1][0])
    else:
        return

# Extract denominator rating or return nothing, meaning we do not change the current rating
def extract_denominator_rating(row):
    match = re.findall('((?:\d+\.)?\d+)\/(\d+)', row['text'])
    if match:
        return float(match[-1][1])
    else:
        return

In [832]:

# Code : apply the previous fonctions to all the rows in the dataframe
# Source : looking for "function every row pandas" on a search engine leads to 
# http://jonathansoma.com/lede/foundations/classes/pandas%20columns%20and%20functions/apply-a-function-to-every-row-in-a-pandas-dataframe/
df_archive_clean['rating_numerator'] = df_archive_clean.apply(extract_numerator_rating, axis=1)
df_archive_clean['rating_denominator'] = df_archive_clean.apply(extract_denominator_rating, axis=1)

In [833]:

# Test: Observe all the rows with denominator != 10
df_archive_clean.query('rating_denominator != 10')[['tweet_id', 'rating_numerator', 'rating_denominator', 'text']]

Out[833]:

	tweet_id	rating_numerator	rating_denominator	text
433	820690176645140481	84.0	70.0	The floofs have been released I repeat the floofs have been released. 84/70 https://t.co/NIYC820tmd
516	810984652412424192	24.0	7.0	Meet Sam. She smiles 24/7 & secretly aspires to be a reindeer. \nKeep Sam smiling by clicking and sharing this link:\nhttps://t.co/98tB8y7y7t https://t.co/LouL5vdvxx
902	758467244762497024	165.0	150.0	Why does this never happen at my front door... 165/150 https://t.co/HmwrdfEfUE
1120	731156023742988288	204.0	170.0	Say hello to this unbelievably well behaved squad of doggos. 204/170 would try to pet all at once https://t.co/yGQI3He3xv
1228	713900603437621249	99.0	90.0	Happy Saturday here's 9 puppers on a bench. 99/90 good work everybody https://t.co/mpvaVxKmc1
1254	710658690886586372	80.0	80.0	Here's a brigade of puppers. All look very prepared for whatever happens next. 80/80 https://t.co/0eb7R1Om12
1274	709198395643068416	45.0	50.0	From left to right:\nCletus, Jerome, Alejandro, Burp, & Titson\nNone know where camera is. 45/50 would hug all at once https://t.co/sedre1ivTK
1351	704054845121142784	60.0	50.0	Here is a whole flock of puppers. 60/50 I'll take the lot https://t.co/9dpcw6MdWa
1433	697463031882764288	44.0	40.0	Happy Wednesday here's a bucket of pups. 44/40 would pet all at once https://t.co/HppvrYuamZ
1635	684222868335505415	121.0	110.0	Someone help the girl is being mugged. Several are distracting her while two steal her shoes. Clever puppers 121/110 https://t.co/1zfnTJLt55
1779	677716515794329600	144.0	120.0	IT'S PUPPERGEDDON. Total of 144/120 ...I think https://t.co/ZanVtAtvIq
1843	675853064436391936	88.0	80.0	Here we have an entire platoon of puppers. Total score: 88/80 would pet all at once https://t.co/y93p6FLvVw

We have one line with a wrong value 24/7. As there is no rating within the associated text, we simply delete the line.

In [834]:

# Delete the line with 24/7 as rating
df_archive_clean.drop([516], inplace=True)

Cleaning - Quality Issue 5 : WeRateDgos twitter archive - We have "None" as name even if, sometimes, there is an available name in the text. So few names are missing and could be retrieved.¶

In [835]:

# List the rows with missing names.
df_nametofind = df_archive_clean.query('name == "None"')[['tweet_id', 'name', 'text']]
df_nametofind

Out[835]:

	tweet_id	name	text
5	891087950875897856	None	Here we have a majestic great white breaching off South Africa's coast. Absolutely h*ckin breathtaking. 13/10 (IG: tucker_marlo) #BarkWeek https://t.co/kQ04fDDRmh
7	890729181411237888	None	When you watch your owner call another dog a good boy but then they turn back to you and say you're a great boy. 13/10 https://t.co/v0nONBcwxq
12	889665388333682689	None	Here's a puppo that seems to be on the fence about something haha no but seriously someone help her. 13/10 https://t.co/BxvuXk0UCm
24	887343217045368832	None	You may not have known you needed to see this today. 13/10 please enjoy (IG: emmylouroo) https://t.co/WZqNqygEyV
25	887101392804085760	None	This... is a Jubilant Antarctic House Bear. We only rate dogs. Please only send dogs. Thank you... 12/10 would suffocate in floof https://t.co/4Ad1jzJSdp
35	885518971528720385	None	I have a new hero and his name is Howard. 14/10 https://t.co/gzLHboL7Sk
37	885167619883638784	None	Here we have a corgi undercover as a malamute. Pawbably doing important investigative work. Zero control over tongue happenings. 13/10 https://t.co/44ItaMubBf
41	884441805382717440	None	I present to you, Pup in Hat. Pup in Hat is great for all occasions. Extremely versatile. Compact as h*ck. 14/10 (IG: itselizabethgales) https://t.co/vvBOcC2VdC
42	884247878851493888	None	OMG HE DIDN'T MEAN TO HE WAS JUST TRYING A LITTLE BARKOUR HE'S SUPER SORRY 13/10 WOULD FORGIVE IMMEDIATE https://t.co/uF3pQ8Wubj
47	883117836046086144	None	Please only send dogs. We don't rate mechanics, no matter how h*ckin good. Thank you... 13/10 would sneak a pat https://t.co/Se5fZ9wp5E
59	880872448815771648	None	Ugh not again. We only rate dogs. Please don't send in well-dressed floppy-tongued street penguins. Dogs only please. Thank you... 12/10 https://t.co/WiAMbTkDPf
62	880095782870896641	None	Please don't send in photos without dogs in them. We're not @porch_rates. Insubordinate and churlish. Pretty good porch tho 11/10 https://t.co/HauE8M3Bu4
72	878604707211726852	None	Martha is stunning how h*ckin dare you. 13/10 https://t.co/9uABQXgjwa
83	876537666061221889	None	I can say with the pupmost confidence that the doggos who assisted with this search are heroic as h*ck. 14/10 for all https://t.co/8yoc1CNTsu
88	875097192612077568	None	You'll get your package when that precious man is done appreciating the pups. 13/10 for everyone https://t.co/PFp4MghzBW
89	875021211251597312	None	Guys please stop sending pictures without any dogs in th- oh never mind hello excuse me sir. 12/10 stealthy as h*ck https://t.co/brCQoqc8AW
93	874057562936811520	None	I can't believe this keeps happening. This, is a birb taking a bath. We only rate dogs. Please only send dogs. Thank you... 12/10 https://t.co/pwY9PQhtP2
96	873580283840344065	None	We usually don't rate Deck-bound Saskatoon Black Bears, but this one is h*ckin flawless. Sneaky tongue slip too. 13/10 would hug firmly https://t.co/mNuMH9400n
99	872967104147763200	None	Here's a very large dog. He has a date later. Politely asked this water person to check if his breath is bad. 12/10 good to go doggo https://t.co/EMYIdoblMR
100	872820683541237760	None	Here are my favorite #dogsatpollingstations \nMost voted for a more consistent walking schedule and to increase daily pats tenfold. All 13/10 https://t.co/17FVMl4VZ5
103	872486979161796608	None	We. Only. Rate. Dogs. Do not send in other things like this fluffy floor shark clearly ready to attack. Get it together guys... 12/10 https://t.co/BZHiKx3FpQ
110	871102520638267392	None	Never doubt a doggo 14/10 https://t.co/AbBLh2FZCH
112	870804317367881728	None	Real funny guys. Sending in a pic without a dog in it. Hilarious. We'll rate the rug tho because it's giving off a very good vibe. 11/10 https://t.co/GCD1JccCyi
125	868622495443632128	None	Here's a h*ckin peaceful boy. Unbothered by the comings and goings. 13/10 please reveal your wise ways https://t.co/yeaH8Ej5eM
127	867900495410671616	None	Unbelievable. We only rate dogs. Please don't send in non-canines like the "I" from Pixar's opening credits. Thank you... 12/10 https://t.co/JMhDNv5wXZ
131	867051520902168576	None	Oh my this spooked me up. We only rate dogs, not happy ghosts. Please send dogs only. It's a very simple premise. Thank you... 13/10 https://t.co/M5Rz0R8SIQ
133	866720684873056260	None	He was providing for his family 13/10 how dare you https://t.co/Q8mVwWN3f4
141	864873206498414592	None	We only rate dogs. Please don't send in Jesus. We're trying to remain professional and legitimate. Thank you... 14/10 https://t.co/wr3xsjeCIR
154	862096992088072192	None	We only rate dogs. Please don't send perfectly toasted marshmallows attempting to drive. Thank you... 13/10 https://t.co/nvZyyrp0kd
157	861288531465048066	None	HI. MY. NAME. IS. BOOMER. AND. I. WANT. TO. SAY. IT'S. H*CKIN. RIDICULOUS. THAT. DOGS. CAN'T VOTE. ABSOLUTE. CODSWALLUP. THANK. YOU. 13/10 https://t.co/SqKJPwbQ2g
...	...	...	...
2288	667176164155375616	None	These are strange dogs. All have toupees. Long neck for dogs. In a shed of sorts? Work in groups? 4/10 still petable https://t.co/PZxSarAfSN
2294	667138269671505920	None	Extremely intelligent dog here. Has learned to walk like human. Even has his own dog. Very impressive 10/10 https://t.co/0DvHAMdA4V
2299	667065535570550784	None	Here we have a Hufflepuff. Loves vest. Eyes wide af. Flaccid tail. Matches carpet. Always a little blurry. 8/10 https://t.co/7JdgVqDnvR
2301	667044094246576128	None	12/10 gimme now https://t.co/QZAnwgnOMB
2305	666837028449972224	None	My goodness. Very rare dog here. Large. Tail dangerous. Kinda fat. Only eats leaves. Doesn't come when called 3/10 https://t.co/xYGdBrMS9h
2306	666835007768551424	None	These are Peruvian Feldspars. Their names are Cupit and Prencer. Both resemble Rand Paul. Sick outfits 10/10 & 10/10 https://t.co/ZnEMHBsAs1
2307	666826780179869698	None	12/10 simply brilliant pup https://t.co/V6ZzG45zzG
2310	666786068205871104	None	Unfamiliar with this breed. Ears pointy af. Won't let go of seashell. Won't eat kibble. Not very fast. Bad dog 2/10 https://t.co/EIn5kElY1S
2316	666649482315059201	None	Cool dog. Enjoys couch. Low monotone bark. Very nice kicks. Pisses milk (must be rare). Can't go down stairs. 4/10 https://t.co/vXMKrJC81s
2320	666437273139982337	None	Here we see a lone northeastern Cumberbatch. Half ladybug. Only builds with bricks. Very confident with body. 7/10 https://t.co/7LtjBS0GPK
2321	666435652385423360	None	"Can you behave? You're ruining my wedding day"\nDOG: idgaf this flashlight tastes good as hell\n\n10/10 https://t.co/GlFZPzqcEU
2322	666430724426358785	None	Oh boy what a pup! Sunglasses take this one to the next level. Weirdly folds front legs. Pretty big. 6/10 https://t.co/yECbFrSArM
2323	666428276349472768	None	Here we have an Austrian Pulitzer. Collectors edition. Levitates (?). 7/10 would garden with https://t.co/NMQq6HIglK
2324	666421158376562688	None	internally screaming 12/10 https://t.co/YMcrXC2Y6R
2328	666396247373291520	None	Oh goodness. A super rare northeast Qdoba kangaroo mix. Massive feet. No pouch (disappointing). Seems alert. 9/10 https://t.co/Dc7b0E8qFE
2329	666373753744588802	None	Those are sunglasses and a jean jacket. 11/10 dog cool af https://t.co/uHXrPkUEyl
2330	666362758909284353	None	Unique dog here. Very small. Lives in container of Frosted Flakes (?). Short legs. Must be rare 6/10 would still pet https://t.co/XMD9CwjEnM
2331	666353288456101888	None	Here we have a mixed Asiago from the Galápagos Islands. Only one ear working. Big fan of marijuana carpet. 8/10 https://t.co/tltQ5w9aUO
2332	666345417576210432	None	Look at this jokester thinking seat belt laws don't apply to him. Great tongue tho 10/10 https://t.co/VFKG1vxGjB
2336	666273097616637952	None	Can take selfies 11/10 https://t.co/ws2AMaNwPW
2337	666268910803644416	None	Very concerned about fellow dog trapped in computer. 10/10 https://t.co/0yxApIikpk
2338	666104133288665088	None	Not familiar with this breed. No tail (weird). Only 2 legs. Doesn't bark. Surprisingly quick. Shits eggs. 1/10 https://t.co/Asgdc6kuLX
2339	666102155909144576	None	Oh my. Here you are seeing an Adobe Setter giving birth to twins!!! The world is an amazing place. 11/10 https://t.co/11LvqN4WLq
2340	666099513787052032	None	Can stand on stump for what seems like a while. Built that birdhouse? Impressive. Made friends with a squirrel. 8/10 https://t.co/Ri4nMTLq5C
2341	666094000022159362	None	This appears to be a Mongolian Presbyterian mix. Very tired. Tongue slip confirmed. 9/10 would lie down with https://t.co/mnioXo3IfP
2342	666082916733198337	None	Here we have a well-established sunblockerspaniel. Lost his other flip-flop. 6/10 not very waterproof https://t.co/3RU6x0vHB7
2343	666073100786774016	None	Let's hope this flight isn't Malaysian (lol). What a dog! Almost completely camouflaged. 10/10 I trust this pilot https://t.co/Yk6GHE9tOY
2344	666071193221509120	None	Here we have a northern speckled Rhododendron. Much sass. Gives 0 fucks. Good tongue. 9/10 would caress sensually https://t.co/ZoL8kq2XFx
2351	666049248165822465	None	Here we have a 1949 1st generation vulpix. Enjoys sweat tea and Fox News. Cannot be phased. 5/10 https://t.co/4B7cOc1EDq
2355	666020888022790149	None	Here we have a Japanese Irish Setter. Lost eye in Vietnam (?). Big fan of relaxing on stair. 8/10 would pet https://t.co/BLDqew2Ijj

600 rows × 3 columns

From a visual observation within the "text", we identify 3 situations:

Two cases where the follows "name is" and "IS."
One case where the name "Martha" starts the text line
All the other cases, the name was really missing within the text

Over all the 600 rows impacted here, as the name is really missing for 597 rows, we do not see the benefit of having a huge regular expression to detect the name. Thus we do not clean this issue. This cleaning will not bring any added value to our further analysis and insights.

Cleaning - Quality Issue 6 : WeRateDgos twitter archive - We have missing dog stages, meaning stages not properly extracted. This is the case for "puppo", "doggo" and "pupper".¶

In [836]:

# First we count the missing stages
print(df_archive_clean.puppo.value_counts())
print(df_archive_clean.doggo.value_counts())
print(df_archive_clean.pupper.value_counts())
print(df_archive_clean.floofer.value_counts())

None     2069
puppo    24  
Name: puppo, dtype: int64
None     2010
doggo    83  
Name: doggo, dtype: int64
None      1864
pupper    229 
Name: pupper, dtype: int64
None       2083
floofer    10  
Name: floofer, dtype: int64

In [837]:

# Define : For each non identified stage, we do through the associated text to find the stage

# Code : function which extract "puppo" from the text
def extract_puppo(row):
    found = row['text'].lower().find("puppo")
    if found != -1:
        return "puppo"
    else:
        return

# Code : function which extract "doggo" from the text
def extract_doggo(row):
    found = row['text'].lower().find("doggo")
    if found != -1:
        return "doggo"
    else:
        return

# Code : function which extract "pupper" from the text
def extract_pupper(row):
    found = row['text'].lower().find("pupper")
    if found != -1:
        return "pupper"
    else:
        return

In [838]:

# Code : apply the detection on the dataset
df_archive_clean['puppo'] = df_archive_clean.apply(extract_puppo, axis=1)
df_archive_clean['doggo'] = df_archive_clean.apply(extract_doggo, axis=1)
df_archive_clean['pupper'] = df_archive_clean.apply(extract_pupper, axis=1)

In [839]:

# Test: we count the missing stages, again
print(df_archive_clean.puppo.value_counts())
print(df_archive_clean.doggo.value_counts())
print(df_archive_clean.pupper.value_counts())

puppo    30
Name: puppo, dtype: int64
doggo    91
Name: doggo, dtype: int64
pupper    254
Name: pupper, dtype: int64

Cleaning - Quality issue 9 : Tweets images predictions - We have rows which are not dogs images predictions.¶

This cleaning include the correction for the

Quality issue 11 : Tweets images predictions - predictions and the associated confidences are spread over several columns.¶

Define : if the p1_dog is not a dog, we get the first from p2 and p3 which are dog prediction. In case, none of them is a dog prediction, we get the most highest confidence prediction, whatever it is. We know we are looking for dogs, so even with a lower confidence, we take the dog detection.

In [840]:

# Code : set prediction - for p1, p2 and p3, we get the prediction from the 1st which is a dog, 
# otherwise we stay on p1 which used to have the highest confidence
def set_prediction(row):
    prediction = row['p1']
    if(row['p1_dog'] == False):
        if(row['p2_dog'] == True):
            prediction = row['p2']
        else:
            if(row['p3_dog'] == True):
                prediction = row['p3']
        
    return prediction

# Code: set confidence - se stay on p1_conf unless we find a dog on p2 or p3,
# and in that case we take the associated confidence
def set_confidence(row):
    confidence = row['p1_conf']
    if(row['p1_dog'] == False):
        if(row['p2_dog'] == True):
            confidence = row['p2_conf']
        else:
            if(row['p3_dog'] == True):
                confidence = row['p3_conf']
                
    return confidence

# Code: set the type - As we might not know all the predictions which are not dogs, 
# we still need to know easily if we have a dog or something else
def set_detectiontype(row):
    detected_type = "dog"
    if(row['p1_dog'] == False):
        if(row['p2_dog'] == False):
            if(row['p3_dog'] == False):
                detected_type = "other"
            
    return detected_type

In [841]:

# Code : create the new columns with the values
df_image_clean['prediction'] = df_image_clean.apply(set_prediction, axis=1)
df_image_clean['confidence'] = df_image_clean.apply(set_confidence, axis=1)
df_image_clean['detectiontype'] = df_image_clean.apply(set_detectiontype, axis=1)

In [842]:

# Test : Visual verification
df_image_clean.query('detectiontype == "other"')[['p1', 'p1_conf', 'p1_dog', 'p2', 'p2_conf', 'p2_dog', 'p3', 'p3_conf', 'p3_dog', 'prediction', 'confidence', 'detectiontype']].sample(10)

Out[842]:

	p1	p1_conf	p1_dog	p2	p2_conf	p2_dog	p3	p3_conf	p3_dog	prediction	confidence	detectiontype
132	shopping_basket	0.398361	False	hamper	0.363222	False	bassinet	0.084173	False	shopping_basket	0.398361	other
1979	home_theater	0.168290	False	sandbar	0.098040	False	television	0.079729	False	home_theater	0.168290	other
1097	alp	0.320126	False	lawn_mower	0.080808	False	viaduct	0.065321	False	alp	0.320126	other
1902	pencil_box	0.662183	False	purse	0.066505	False	pillow	0.044725	False	pencil_box	0.662183	other
453	seashore	0.352321	False	promontory	0.131753	False	wreck	0.095597	False	seashore	0.352321	other
1036	espresso	0.430135	False	coffee_mug	0.418483	False	cup	0.088391	False	espresso	0.430135	other
1142	doormat	0.359586	False	china_cabinet	0.053901	False	passenger_car	0.052665	False	doormat	0.359586	other
1937	lakeside	0.312299	False	dock	0.159842	False	canoe	0.070795	False	lakeside	0.312299	other
45	snail	0.999888	False	slug	0.000055	False	acorn	0.000026	False	snail	0.999888	other
832	washbasin	0.272451	False	doormat	0.165871	False	bathtub	0.066368	False	washbasin	0.272451	other

In [843]:

# Clean the p columns
df_image_clean.drop('p1', axis=1, inplace=True)
df_image_clean.drop('p1_conf', axis=1, inplace=True)
df_image_clean.drop('p1_dog', axis=1, inplace=True)
df_image_clean.drop('p2', axis=1, inplace=True)
df_image_clean.drop('p2_conf', axis=1, inplace=True)
df_image_clean.drop('p2_dog', axis=1, inplace=True)
df_image_clean.drop('p3', axis=1, inplace=True)
df_image_clean.drop('p3_conf', axis=1, inplace=True)
df_image_clean.drop('p3_dog', axis=1, inplace=True)

In [844]:

# View on the dataset
df_image_clean.sample(5)

Out[844]:

	tweet_id	jpg_url	img_num	prediction	confidence	detectiontype
689	684188786104872960	https://pbs.twimg.com/media/CX66EiJWkAAVjA-.jpg	1	kelpie	0.537782	dog
972	706644897839910912	https://pbs.twimg.com/ext_tw_video_thumb/706644797256241152/pu/img/NTqvmIUQExGmKFSR.jpg	1	Chihuahua	0.132928	dog
233	670420569653809152	https://pbs.twimg.com/media/CU3P82RWEAAIVrE.jpg	1	bow_tie	0.268759	other
1867	843856843873095681	https://pbs.twimg.com/media/C7X7Ui0XgAA3m19.jpg	1	Labrador_retriever	0.922540	dog
488	675517828909424640	https://pbs.twimg.com/media/CV_r3v4VAAALvwg.jpg	1	Scottish_deerhound	0.240591	dog

The "detectiontype" column is not really necessary here. We added it to have a quick and easy understanding about the prediction, if it is a kind of dog or not. It might be useful for the analysis.

Clean Tidiness issue 1: WeRateDgos twitter archive - Dog stages have been spread as columns.¶

In [845]:

# Define : for each doggo, floofer, pupper and puppo columns, we get the stage we encounter
# In case we got a stage earlier, we add collect them separated by a ","
# Code : Get the stage from each column
def set_stage(row):
    stage = None
    
    # Collect doggo and any other stage encoutered separated by a ","
    if(row['doggo'] == "doggo"):
        if stage != None:
            stage = stage + "," + "floofer"
            print("more floofer : " + stage)
        else:
            stage = "doggo"
    
    # Collect floofer and any other stage encoutered separated by a ","
    if(row['floofer'] == "floofer"):
            
        if stage != None:
            stage = stage + "," + "floofer"
            print("more floofer : " + stage)
        else:
            stage = "floofer"
    
    # Collect pupper and any other stage encoutered separated by a ","
    if(row['pupper'] == "pupper"):
                
        if stage != None:
            stage = stage + "," + "pupper"
            print("more pupper : " + stage)
        else:
            stage = "pupper"
    
    # Collect puppo and any other stage encoutered separated by a ","
    if(row['puppo'] == "puppo"):
                    
        if stage != None:
            stage = stage + "," + "puppo"
            print("more puppo : " + stage)
        else:
            stage = "puppo"
                
    return stage

In [846]:

# Code : set the new column using the previous functions
df_archive_clean['stage'] = df_archive_clean.apply(set_stage, axis=1)

more puppo : doggo,puppo
more puppo : doggo,puppo
more floofer : doggo,floofer
more pupper : doggo,pupper
more pupper : doggo,pupper
more pupper : doggo,pupper
more pupper : doggo,pupper
more pupper : doggo,pupper
more pupper : doggo,pupper
more pupper : doggo,pupper
more pupper : doggo,pupper
more pupper : doggo,pupper

In [847]:

# Test : get the unique values from "stage" column
df_archive_clean['stage'].unique()

Out[847]:

array([None, 'doggo', 'puppo', 'pupper', 'floofer', 'doggo,puppo',
       'doggo,floofer', 'doggo,pupper'], dtype=object)

In [848]:

# Clean the non useful columns
df_archive_clean.drop('puppo', axis=1, inplace=True)
df_archive_clean.drop('doggo', axis=1, inplace=True)
df_archive_clean.drop('floofer', axis=1, inplace=True)
df_archive_clean.drop('pupper', axis=1, inplace=True)

Cleaning - Quality Issue 7 : WeRateDgos twitter archive - "expanded_urls" column contains duplicated urls.¶

In [849]:

# Define : we keep only the 1st url in "expanded_urls". 
# The reason: From previous observation, we have rare cases where the urls are differents for the same tweet.
# We do not plan to use the 2nd or the 3rd urls for our analysis

# Code : function to split the current "expanded_urls" content and keep the 1st part
def split_expanded_urls(row):
    retrieved_url = None
    found_urls = row['expanded_urls'].split(',')
    retrieved_url = found_urls[0]
    
    return retrieved_url

In [850]:

# Code : apply the function above
df_archive_clean['expanded_urls'] = df_archive_clean.apply(split_expanded_urls, axis=1)

In [851]:

# Test : Do we still have multiple urls ?
df_archive_clean['expanded_urls'].str.contains(',').value_counts()

Out[851]:

False    2093
Name: expanded_urls, dtype: int64

Cleaning - Quality Issue 8 : WeRateDgos twitter archive - timestamp is using string type.¶

In [852]:

# Define and Code: Convert timestamp to datetime using pd.to_datetime function
df_archive_clean['timestamp'] = pd.to_datetime(df_archive_clean['timestamp'])

In [853]:

# Test: get the "timestamp" column type
type(df_archive_clean['timestamp'][0])

Out[853]:

pandas._libs.tslibs.timestamps.Timestamp

Cleaning - Quality Issue 10 : Data retrieved via twitter API - "created_at" column type is not timestamp.¶

In [854]:

# Define and Code: Convert timestamp to datetime using pd.to_datetime function
df_twitter_clean['created_at'] = pd.to_datetime(df_twitter_clean['created_at'])

In [855]:

# Test: get the "created_at" column type
type(df_twitter_clean['created_at'][0])

Out[855]:

pandas._libs.tslibs.timestamps.Timestamp

Requirement : we do not keep the tweets beyond August 1st 2017, because we won't be able to have the associated images predictions¶

In [856]:

# twitter enhanced archive : Which lines are beyond August 1st 2017
df_archive_clean.query('timestamp > "2017-08-01 00:00:00"')[['tweet_id', 'timestamp']]

Out[856]:

	tweet_id	timestamp
0	892420643555336193	2017-08-01 16:23:56
1	892177421306343426	2017-08-01 00:17:27

In [857]:

# Delete the line
df_archive_clean.drop([0], inplace=True)
df_archive_clean.drop([1], inplace=True)

In [858]:

# Test : ensure the removal
df_archive_clean.query('timestamp > "2017-08-01 00:00:00"')[['tweet_id', 'timestamp']]

Out[858]:

	tweet_id	timestamp

In [859]:

# Dataframe from twitter AP : Which lines are beyond August 1st 2017
df_twitter_clean.query('created_at > "2017-08-01 00:00:00"')[['tweet_id', 'created_at']]

Out[859]:

	tweet_id	created_at
0	892420643555336193	2017-08-01 16:23:56
1	892177421306343426	2017-08-01 00:17:27

In [860]:

# Delete the line
df_twitter_clean.drop([0], inplace=True)
df_twitter_clean.drop([1], inplace=True)

In [861]:

# Check the removal
df_twitter_clean.query('created_at > "2017-08-01 00:00:00"')[['tweet_id', 'created_at']]

Out[861]:

	tweet_id	created_at

Cleaning Tidiness issue 2 : Merging the twitter enhanced archive dataframe, the twitter dataframe built from the APIs and the images predictions dataframe¶

Merge df_archive_clean with df_twitter_clean, and then merge the result with df_image_clean

In [862]:

# Get the sizes of off the dataframes to merge
df_archive_clean.shape, df_twitter_clean.shape, df_image_clean.shape

Out[862]:

((2091, 9), (2335, 5), (2075, 6))

In [863]:

# This approach does the job, but some rows disappeared (1962 rows as result) during the merge and we don't understand why
# so we abandon it
# https://stackoverflow.com/questions/23668427/pandas-three-way-joining-multiple-dataframes-on-columns
# Merge 3 dataframes 

# Import the python reduce function from the functools package
#from functools import reduce

# Set the list of the dataframes we want to merge
#datasets_list = [df_image_clean, df_twitter_clean[['tweet_id', 'retweet_count', 'favorite_count']], df_archive_clean]

# Merge to create df_master
#df_master = reduce(lambda left,right: pd.merge(left,right,on='tweet_id'), datasets_list)

# Check the size
#df_master.shape

In [864]:

# Code : Merge df_archive_clean and df_twitter_clean
df_temp_tweets = df_archive_clean.merge(df_twitter_clean[['tweet_id', 'retweet_count', 'favorite_count']], how='left', on='tweet_id')

In [865]:

# Test : we should have the same number of rows as df_archive_clean
df_temp_tweets.shape

Out[865]:

(2091, 11)

In [866]:

# Code : merge the temporary dataframe with df_image_clean
df_master = df_image_clean.merge(df_temp_tweets, how='left', on='tweet_id')

In [867]:

# Test: check the size
df_master.shape

Out[867]:

(2075, 16)

In [868]:

# Check df_master's columns names and types
df_master.dtypes

Out[868]:

tweet_id              int64         
jpg_url               object        
img_num               int64         
prediction            object        
confidence            float64       
detectiontype         object        
timestamp             datetime64[ns]
source                object        
text                  object        
expanded_urls         object        
rating_numerator      float64       
rating_denominator    float64       
name                  object        
stage                 object        
retweet_count         float64       
favorite_count        float64       
dtype: object

In [869]:

# For our analysis, we prefer to have only one column for the rating

# Calculate the rating by rating_numerator / rating_denominator, as a float
def calculate_rating(row):
    return float(row['rating_numerator']/row['rating_denominator'])

# Apply the rating function to each row
df_master['rating'] = df_master.apply(calculate_rating, axis=1)

In [870]:

# Remove rating numerator and denominator columns
df_master.drop('rating_numerator', axis=1, inplace=True)
df_master.drop('rating_denominator', axis=1, inplace=True)

In [871]:

# Observe the new dataframe
df_master.sample(2)

Out[871]:

	tweet_id	jpg_url	img_num	prediction	confidence	detectiontype	timestamp	source	text	expanded_urls	name	stage	retweet_count	favorite_count	rating
617	680206703334408192	https://pbs.twimg.com/media/CXCUYcRW8AAObYM.jpg	1	Christmas_stocking	0.149758	other	2015-12-25 02:01:30	<a href="http://twitter.com/download/iphone" rel="nofollow">Twitter for iPhone</a>	I hope everyone enjoys this picture as much as I do. This is Toby. 12/10 https://t.co/vHnu1g9EJm	https://twitter.com/dog_rates/status/680206703334408192/photo/1	Toby	None	1246.0	2941.0	1.2
4	666049248165822465	https://pbs.twimg.com/media/CT5IQmsXIAAKY4A.jpg	1	miniature_pinscher	0.560311	dog	2015-11-16 00:24:50	<a href="http://twitter.com/download/iphone" rel="nofollow">Twitter for iPhone</a>	Here we have a 1949 1st generation vulpix. Enjoys sweat tea and Fox News. Cannot be phased. 5/10 https://t.co/4B7cOc1EDq	https://twitter.com/dog_rates/status/666049248165822465/photo/1	None	None	42.0	105.0	0.5

Storing Cleaned Data¶

In [872]:

# Store cleaned data into csv files
df_master.to_csv('twitter_archive_master.csv')

Store the dataframes into SQLite database.
We've learnt the way to do from here : https://stackoverflow.com/questions/50803109/how-to-store-pandas-dataframe-in-sqlite-db#50803252

In [873]:

# in case the databse already exists
!rm weratedogs.sqlite

In [874]:

# Import the required package
from sqlalchemy import create_engine

# Create the engine - in memory
engine = create_engine('sqlite:///weratedogs.sqlite', echo=False)

# Store our dataframes into 2 tables
df_master.to_sql('twitter_archive_master', con=engine)

In [875]:

# Test: select information from the tables
# Send the request
my_request = engine.execute("SELECT twitter_archive_master.name, twitter_archive_master.rating FROM twitter_archive_master WHERE twitter_archive_master.rating > 1.4")

# Display the response
for name, rating in my_request:
    print(name, rating)

None 42.0
Atticus 177.6

Analyzing and Visualizing¶

We would like to develop an understanding of what make a dog picture to be a success.
We use categorical plots, as discovered from here : https://seaborn.pydata.org/tutorial/categorical.html#categorical-scatterplots

First, we would like to understand the most successful stages

In [876]:

# make a copy of the master dataset
df_stages = df_master.copy()

Each cell in the stage column contains several values separated by a ," character. We need to have one stage per line. Here is the approach we use:

for each cell we remove any "," character at the begining or at the end
we split the content according to the "," character we encounter, in order to have one column per stage
we stack all the stages columns into one single column. So for each stage, we get additional lines into the dataset
we rename the new column to stacked_stages
we add this new column to the original dataset

This way of slipping and stacking has been obtained from the Udacity's student forum, while working on a previous project.

In [877]:

# stack the copy on stage column, and get all the stages in a new column
df_stages = df_stages.join(df_stages.stage.str.strip(',').str.split(',', expand=True).stack().reset_index(level=1, drop=True).rename('stacked_stages'))

In [878]:

# Do we have unique stage into df_stages ?
df_stages.stacked_stages.value_counts()

Out[878]:

pupper     233
doggo      79 
puppo      29 
floofer    8  
Name: stacked_stages, dtype: int64

In [879]:

# Plot favourite_count per stage, in swarn

# visibility
myplot = sns.catplot(x="stacked_stages", y="favorite_count", kind="swarm", data=df_stages, height=8)
# Set the titles
myplot.set(title='Popular stages along with favourites counts', xlabel='Dogs Stages', ylabel='Number of favourites');

Insight 1 :¶

Beyond 80 000 favourites count, we have more doggo(3) than puppo(1) and pupper(1). So it seems that it is better be a doggo to the road to the success here !

In [880]:

# Most stages in 2015
df_stages.query('timestamp > "2015-01-01 00:00:00" and timestamp <= "2016-01-01 00:00:00"')['stacked_stages'].value_counts()

Out[880]:

pupper    73
Name: stacked_stages, dtype: int64

In [881]:

# Most stages in 2016
df_stages.query('timestamp > "2016-01-01 00:00:00" and timestamp <= "2017-01-01 00:00:00"')['stacked_stages'].value_counts()

Out[881]:

pupper     138
doggo      46 
puppo      13 
floofer    6  
Name: stacked_stages, dtype: int64

Doggo stage appears in 2015, but the majority of the stages were still pupper.

In [882]:

# Most stages in 2017
df_stages.query('timestamp > "2017-01-01 00:00:00"')['stacked_stages'].value_counts()

Out[882]:

doggo      33
pupper     22
puppo      16
floofer    2 
Name: stacked_stages, dtype: int64

Then doggo is on the rise.

Insight 2 :¶

Clearly we have more pupper pictures posted, because pupper pictures have been posted during 2015. This is strange, but might be linked to people awareness regarding the WeRateDogs twitter account, at the beginning.

Does the rating correlated to the most favorited ?

In [883]:

# Get the rating of the most favorited
df_most_favorited = df_stages.query('favorite_count > 80000')
df_most_favorited.rating.value_counts()

Out[883]:

1.3    7
1.2    2
1.4    1
Name: rating, dtype: int64

In [884]:

# How is this rating positionned among all the others ?
df_master.describe()

Out[884]:

	tweet_id	img_num	confidence	retweet_count	favorite_count	rating
count	2.075000e+03	2075.000000	2075.000000	1964.000000	1964.000000	1968.000000
mean	7.384514e+17	1.203855	0.547955	2625.843177	8642.184318	1.163556
std	6.785203e+16	0.561875	0.297842	4693.838058	12689.882348	4.090735
min	6.660209e+17	1.000000	0.000010	11.000000	77.000000	0.000000
25%	6.764835e+17	1.000000	0.299295	584.750000	1854.750000	1.000000
50%	7.119988e+17	1.000000	0.541780	1259.000000	3911.000000	1.100000
75%	7.932034e+17	1.000000	0.820962	3004.250000	10806.750000	1.200000
max	8.924206e+17	4.000000	1.000000	82678.000000	162549.000000	177.600000

Insight 3 :¶

The most favorited tweets (meaning images) are rated high, more than the rating and in the last quartile up to the highest rate.
So, basically the most favorited tweets are highly rated, which is a normal expected situation here.

In [885]:

# Are they part of the most retweeted ?
retweetplot = sns.catplot(x="stacked_stages", y="retweet_count", kind="bar", data=df_most_favorited)
retweetplot.set(title='Retweet count for the most favorited tweets', xlabel='Dogs Stages', ylabel='Number of retweets');

/Users/jlcossi/anaconda/lib/python3.6/site-packages/scipy/stats/stats.py:1713: FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use `arr[tuple(seq)]` instead of `arr[seq]`. In the future this will be interpreted as an array index, `arr[np.array(seq)]`, which will result either in an error or a different result.
  return np.add.reduce(sorted[indexer] * weights, axis=axis) / sumval

Insight 4 :¶

We observe this plot along with the statistics summary above. We confirm that the most favorited are well positioned within the most retweeted.

What are the breeds associated to the success ?

In [886]:

# Observe the breeds
breedplot = sns.catplot(x="prediction", y="favorite_count", hue="stacked_stages", kind="bar", data=df_most_favorited, height=14);
breedplot.set(title='Favorited tweets and their breeds', xlabel='Dogs predicted breeds', ylabel='Number of favorited');

We do not have all the stages for the identified breeds here.
Even with this small limitation, we can conclude that being a Labrador_retriever, an Eskimo_dog, a standard_poodle and a doggo might lead to a chance to be part of the most favorited by WeRateDogs members.

We observe in the dataset, that 2 of the 3 successful doggos were shared as videos. So this is an important factor because videos transmit emotions. So there might me other elements which are not in the scope of this study: colors and the quality of the images or videos.

In [887]:

# The favorited tweets, including the breeds
df_most_favorited

Out[887]:

	tweet_id	jpg_url	img_num	prediction	confidence	detectiontype	timestamp	source	text	expanded_urls	name	stage	retweet_count	favorite_count	rating	stacked_stages
571	678399652199309312	https://pbs.twimg.com/ext_tw_video_thumb/678399528077250560/pu/img/BOjUNHRsYLeSo0hl.jpg	1	Bedlington_terrier	0.015047	dog	2015-12-20 02:20:55	<a href="http://twitter.com/download/iphone" rel="nofollow">Twitter for iPhone</a>	This made my day. 12/10 please enjoy https://t.co/VRTbo3aAcm	https://twitter.com/dog_rates/status/678399652199309312/video/1	None	None	33355.0	81543.0	1.2	NaN
1186	739238157791694849	https://pbs.twimg.com/ext_tw_video_thumb/739238016737267712/pu/img/-tLpyiuIzD5zR1et.jpg	1	Eskimo_dog	0.503372	dog	2016-06-04 23:31:25	<a href="http://twitter.com/download/iphone" rel="nofollow">Twitter for iPhone</a>	Here's a doggo blowing bubbles. It's downright legendary. 13/10 would watch on repeat forever (vid by Kent Duryee) https://t.co/YcXgHfp1EC	https://twitter.com/dog_rates/status/739238157791694849/video/1	None	doggo	61211.0	120128.0	1.3	doggo
1221	744234799360020481	https://pbs.twimg.com/ext_tw_video_thumb/744234667679821824/pu/img/1GaWmtJtdqzZV7jy.jpg	1	Labrador_retriever	0.825333	dog	2016-06-18 18:26:18	<a href="http://twitter.com/download/iphone" rel="nofollow">Twitter for iPhone</a>	Here's a doggo realizing you can stand in a pool. 13/10 enlightened af (vid by Tina Conrad) https://t.co/7wE9LTEXC4	https://twitter.com/dog_rates/status/744234799360020481/video/1	None	doggo	82678.0	162549.0	1.3	doggo
1641	807106840509214720	https://pbs.twimg.com/ext_tw_video_thumb/807106774843039744/pu/img/8XZg1xW35Xp2J6JW.jpg	1	Chihuahua	0.505370	dog	2016-12-09 06:17:20	<a href="http://twitter.com/download/iphone" rel="nofollow">Twitter for iPhone</a>	This is Stephan. He just wants to help. 13/10 such a good boy https://t.co/DkBYaCAg2d	https://twitter.com/dog_rates/status/807106840509214720/video/1	Stephan	None	60299.0	125650.0	1.3	NaN
1715	819004803107983360	https://pbs.twimg.com/media/C12whDoVEAALRxa.jpg	1	standard_poodle	0.351308	dog	2017-01-11 02:15:36	<a href="http://twitter.com/download/iphone" rel="nofollow">Twitter for iPhone</a>	This is Bo. He was a very good First Doggo. 14/10 would be an absolute honor to pet https://t.co/AdPKrI8BZ1	https://twitter.com/dog_rates/status/819004803107983360/photo/1	Bo	doggo	39660.0	91294.0	1.4	doggo
1744	822872901745569793	https://pbs.twimg.com/media/C2tugXLXgAArJO4.jpg	1	Lakeland_terrier	0.196015	dog	2017-01-21 18:26:02	<a href="http://twitter.com/download/iphone" rel="nofollow">Twitter for iPhone</a>	Here's a super supportive puppo participating in the Toronto #WomensMarch today. 13/10 https://t.co/nTz3FtorBc	https://twitter.com/dog_rates/status/822872901745569793/photo/1	None	puppo	47136.0	138899.0	1.3	puppo
1932	859196978902773760	https://pbs.twimg.com/ext_tw_video_thumb/859196962498805762/pu/img/-yBpr4-o4GJZECYE.jpg	1	malamute	0.216163	dog	2017-05-02 00:04:57	<a href="http://twitter.com/download/iphone" rel="nofollow">Twitter for iPhone</a>	We only rate dogs. This is quite clearly a smol broken polar bear. We'd appreciate if you only send dogs. Thank you... 12/10 https://t.co/g2nSyGenG9	https://twitter.com/dog_rates/status/859196978902773760/video/1	quite	None	30403.0	89782.0	1.2	NaN
1961	866450705531457537	https://pbs.twimg.com/media/DAZAUfBXcAAG_Nn.jpg	2	French_bulldog	0.905334	dog	2017-05-22 00:28:40	<a href="http://twitter.com/download/iphone" rel="nofollow">Twitter for iPhone</a>	This is Jamesy. He gives a kiss to every other pupper he sees on his walk. 13/10 such passion, much tender https://t.co/wk7TfysWHr	https://twitter.com/dog_rates/status/866450705531457537/photo/1	Jamesy	pupper	35037.0	120747.0	1.3	pupper
1977	870374049280663552	https://pbs.twimg.com/media/DBQwlFCXkAACSkI.jpg	1	golden_retriever	0.841001	dog	2017-06-01 20:18:38	<a href="http://twitter.com/download/iphone" rel="nofollow">Twitter for iPhone</a>	This is Zoey. She really likes the planet. Would hate to see willful ignorance and the denial of fairly elemental science destroy it. 13/10 https://t.co/T1xlgaPujm	https://twitter.com/dog_rates/status/870374049280663552/photo/1	Zoey	None	25805.0	81213.0	1.3	NaN
2014	879415818425184262	https://pbs.twimg.com/ext_tw_video_thumb/879415784908390401/pu/img/cX7XI1TnUsseGET5.jpg	1	English_springer	0.383404	dog	2017-06-26 19:07:24	<a href="http://twitter.com/download/iphone" rel="nofollow">Twitter for iPhone</a>	This is Duddles. He did an attempt. 13/10 someone help him (vid by Georgia Felici) https://t.co/UDT7ZkcTgY	https://twitter.com/dog_rates/status/879415818425184262/video/1	Duddles	None	42962.0	103018.0	1.3	NaN

In [888]:

# Generate the HTML version of this notebook
from subprocess import call
call(['python', '-m', 'nbconvert', 'Wrangle_act.ipynb'])

Out[888]:

In [ ]:

Wrangle and Analyze Data¶

Table of Contents¶

Introduction¶

Gathering¶

1. Gather data from the archive available as twitter-archive-enhanced.csv¶

2. Gather the tweet image predictions¶

3. Gather additional informations directly from twitter¶

Assessing¶

1. The WeRateDogs twitter archive¶

Quality Issue 1 : The WeRateDgos twitter archive enhanced contains retweets, which could not be part or our study¶

Quality Issue 2 : The WeRateDgos twitter archive enhanced has 59 missing value from the "expanded_urls" column¶

Quality Issue 4 : WeRateDgos twitter archive - in some cases, the relevant rating information, consequently rating_numerator and rating_denominator, have not been extracted.¶

Quality Issue 5 : WeRateDgos twitter archive - We have "None" as name even if, sometimes, there is an available name in the text. So few names are missing and could be retrieved.¶

Quality Issue 6 : WeRateDgos twitter archive - We have missing dog stages, meaning stages not properly extracted. This is the case for "puppo", "doggo" and "pupper".¶

Quality Issue 7: WeRateDgos twitter archive - "expanded_urls" column contains duplicated urls¶

Quality Issue 8: WeRateDgos twitter archive - timestamp is using string type.¶

Tidiness Issue 1 : WeRateDgos twitter archive - Dog stages have been spread as columns.¶

2. The tweet image predictions¶

Quality issue 11 : Tweets images predictions - predictions and the associated confidences are spread over several columns.¶

Tidiness Issue 2 : Tweets information spreaded over several files and dataframes.¶

Quality Issue 9 : Tweets images predictions - We have rows which are not dogs images predictions¶

3. Data retrieved via twitter API¶

Quality Issue 10 : Data retrieved via twitter API - "created_at" column type is not timestamp¶

4. Synthesis¶

Tidiness issues identified :¶

Quality issues identified :¶

Cleaning¶

Approach¶

Cleaning - Quality Issue 1 : The WeRateDgos twitter archive enhanced contains retweets, which could not be part or our study¶

Cleaning - Quality Issue 2 : The WeRateDgos twitter archive enhanced has 59 missing value from the "expanded_urls" column¶

Cleaning - Quality Issue 4 : WeRateDgos twitter archive - in some cases, the relevant rating information, consequently rating_numerator and rating_denominator, have not been extracted.¶

Cleaning - Quality Issue 5 : WeRateDgos twitter archive - We have "None" as name even if, sometimes, there is an available name in the text. So few names are missing and could be retrieved.¶

Cleaning - Quality Issue 6 : WeRateDgos twitter archive - We have missing dog stages, meaning stages not properly extracted. This is the case for "puppo", "doggo" and "pupper".¶

Cleaning - Quality issue 9 : Tweets images predictions - We have rows which are not dogs images predictions.¶

Quality issue 11 : Tweets images predictions - predictions and the associated confidences are spread over several columns.¶

Clean Tidiness issue 1: WeRateDgos twitter archive - Dog stages have been spread as columns.¶

Cleaning - Quality Issue 7 : WeRateDgos twitter archive - "expanded_urls" column contains duplicated urls.¶

Cleaning - Quality Issue 8 : WeRateDgos twitter archive - timestamp is using string type.¶

Cleaning - Quality Issue 10 : Data retrieved via twitter API - "created_at" column type is not timestamp.¶

Requirement : we do not keep the tweets beyond August 1st 2017, because we won't be able to have the associated images predictions¶

Cleaning Tidiness issue 2 : Merging the twitter enhanced archive dataframe, the twitter dataframe built from the APIs and the images predictions dataframe¶

Storing Cleaned Data¶

Analyzing and Visualizing¶

Insight 1 :¶

Insight 2 :¶

Insight 3 :¶

Insight 4 :¶

1. Gather data from the archive available as `twitter-archive-enhanced.csv`¶