Hacker News is a site started by the startup incubator Y Combinator, where user-submitted stories (known as "posts") are voted and commented upon, similar to Reddit. Hacker News is extremely popular in technology and startup circles, and posts that make it to the top of Hacker News' listings can get hundreds of thousands of visitors as a result. The main goal of our project is to determine what kind of posts attract more attention and when is it better to post on that website to get noticed.
The dataset was taken here.
I was specifically interested in posts whose titles begin with either Ask HN
or Show HN
. Users submit Ask HN
posts to ask the Hacker News community a specific question. Likewise, users submit Show HN
posts to show the Hacker News community a project, product, or just generally something interesting.
I'll compare these two types of posts to determine the following:
from csv import reader
# open and read the file as a list of lists
opened_file = open("hacker_news.csv")
read_file = reader(opened_file)
hn_raw = list(read_file)
# separate and display the row containing the column header
header = hn_raw[0]
hn_raw = hn_raw[1:]
print("Headers of the table: ", header, "\n")
print("Total number of rows in dataset is: {}".format(len(hn_raw)))
Headers of the table: ['id', 'title', 'url', 'num_points', 'num_comments', 'author', 'created_at'] Total number of rows in dataset is: 293119
The original dataset includes 293119 rows and 7 columns.
Below are descriptions of the columns of the dataset:
id
: The unique identifier from Hacker News for the posttitle
: The title of the posturl
: The URL that the posts links to, if the post has a URLnum_points
: The number of points the post acquired, calculated as the total number of upvotes minus the total number of downvotesnum_comments
: The number of comments that were made on the postauthor
: The username of the person who submitted the postcreated_at
: The date and time at which the post was submittedNext step is to leave only rows with posts with at least one comment and separate posts beginning with Ask HN and Show HN.
# leave only rows with posts which had some comments
hn = []
for row in hn_raw:
numb_comments = row[4]
if numb_comments != "0":
hn.append(row)
print("Number of commented posts in dataset is: {}".format(len(hn)))
# separate posts and append them to lists
ask_posts = []
show_posts = []
other_posts = []
for row in hn:
title = row[1]
if title.lower().startswith("ask hn"):
ask_posts.append(row)
elif title.lower().startswith("show hn"):
show_posts.append(row)
else:
other_posts.append(row)
percentage_to_total_ask = round((len(ask_posts)/len(hn))*100, 2)
percentage_to_total_show = round((len(show_posts)/len(hn))*100, 2)
print("There are {} posts Ask HN".format(len(ask_posts)))
print("It is {}% of the total posts Ask HN with at least one comment".format(percentage_to_total_ask))
print("There are {} posts Show HN".format(len(show_posts)))
print("It is {}% of the total posts Show HN with at least one comment".format(percentage_to_total_show))
print("There are {} other posts".format(len(other_posts)))
Number of commented posts in dataset is: 80401 There are 6911 posts Ask HN It is 8.6% of the total posts Ask HN with at least one comment There are 5059 posts Show HN It is 6.29% of the total posts Show HN with at least one comment There are 68431 other posts
As we can see, there are more posts Ask HN as Show HN: 8.6% versus 6.29% of the total number of posts with at least one comment. Next step is to idenitfy which type of posts attracts more attention.
# determine if ask posts or show posts receive more comments on average
ask_comments = 0
show_comments = 0
# calculate average number of comments for Ask HN
for row in ask_posts:
num_comments = int(row[4])
ask_comments += num_comments
avg_ask_comments = round(ask_comments/len(ask_posts))
# calculate average number of comments for Show HN
for row in show_posts:
num_comments = int(row[4])
show_comments += num_comments
avg_show_comments = round(show_comments/len(show_posts), 2)
# print the result
if avg_ask_comments > avg_show_comments:
print("On average Ask HN posts get more comments as Show HN posts: {} versus {}".format(avg_ask_comments, avg_show_comments))
else:
print("On average Show HN posts get more comments as Ask HN posts: {} versus {}".format(avg_show_comments, avg_ask_comments))
On average Ask HN posts get more comments as Show HN posts: 14 versus 9.81
Ask HN posts get on average more comments as Show HN posts: 14% versus 9.81%
Further I'll focus only on posts Ask HN
I'll determine if ask posts created at a certain time are more likely to attract comments. At first I'll calculate the amount of ask posts created in each hour of the day, along with the number of comments received.
import datetime as dt
#count how many posts were created at any hour and how much comments did they get
comments_by_hour = {}
for row in ask_posts:
date_str = row[6]
num_comments = int(row[4])
# parse date from created_at column and extract hours
date_1 = dt.datetime.strptime(date_str, "%m/%d/%Y %H:%M")
hour = date_1.strftime("%H")
comments_by_hour.setdefault(hour, []).append(num_comments)
print(len(comments_by_hour))
24
# calculate average
for hour, num_comments in comments_by_hour.items():
avg_comments = round(sum(num_comments)/len(num_comments))
comments_by_hour[hour] = avg_comments
# sort and print resulting dictionary
for hour in sorted(comments_by_hour, key=comments_by_hour.get, reverse=True):
print(hour, ':', comments_by_hour[hour])
15 : 40 13 : 22 12 : 15 17 : 14 10 : 14 02 : 13 14 : 13 04 : 13 22 : 12 08 : 12 21 : 11 11 : 11 16 : 11 20 : 11 18 : 11 05 : 11 07 : 10 03 : 10 00 : 10 01 : 9 19 : 9 06 : 9 09 : 8 23 : 8
I identified the hours when Ask HN posts draws more attention: top result is 15:00, followed by 13:00 and 12:00.
As my analysis has showed that the most suitable time to write a Ask post on Hacker News is around 15:00, or following the dataset documentation 3:00 pm est. The longer time span to post - from 01 pm to 03 pm est.