Create A Twitter Application And Use R To Download Data ✓ Solved

Create a Twitter application and use R to download a total o

Create a Twitter application and use R to download a total of 200 tweets: 100 tweets collected by searching a student-chosen hashtag/keyword and 100 tweets collected from a student-chosen Twitter user profile. Deliverables: A word-processing or PDF file explaining the steps to create the Twitter application; an RStudio file containing the R code used to collect the tweets.

Paper For Above Instructions

Overview

This paper describes a reproducible workflow to create a Twitter developer application and use R to collect 200 tweets—100 by hashtag/keyword search and 100 from a specified user timeline. The guidance emphasizes secure credential handling, recommended R packages, common pitfalls (rate limits and sampling biases), and a concise example R workflow. The approach follows best practices from the rtweet package and Twitter developer documentation (Kearney, 2019; Twitter Developer, 2020).

Step 1: Register a Twitter Developer Account and Create an App

1. Apply for a Twitter developer account at the Twitter Developer Portal and accept the developer terms. Choose the appropriate use-case and access level (e.g., Elevated or Academic Research if eligible) because elevated access affects collection limits and endpoints (Twitter Developer, 2020).

2. Create a Project and App in the portal. Note the credentials the portal provides: API Key, API Secret Key, Access Token, Access Token Secret, and Bearer Token. Keep these confidential and do not hard-code them into scripts that may be shared publicly (Kearney, 2019).

Step 2: Prepare R and Packages

1. Install R (R Core Team, 2020) and RStudio. Use packages designed for Twitter API access and HTTP/OAuth handling, notably rtweet (Kearney, 2019) and httr (Wickham, 2019). For text processing, install tidytext or text mining tools such as the tidyverse and tidytext (Silge & Robinson, 2017).

2. Recommended packages: rtweet, httr, jsonlite, dplyr, tidytext, and readr. Keep packages up-to-date to reflect API changes (Kearney, 2019).

Step 3: Securely Store Credentials

1. Avoid embedding keys directly in scripts. Store credentials in environment variables (e.g., in .Renviron) or use key management solutions. Example environment variables: TWITTER_API_KEY, TWITTER_API_SECRET, TWITTER_ACCESS_TOKEN, TWITTER_ACCESS_SECRET, TWITTER_BEARER_TOKEN.

2. Use rtweet authentication helpers to read from environment variables and create an OAuth token programmatically (Kearney, 2019). This approach minimizes accidental credential leaks and is best practice for sharing R scripts without exposing secrets.

Step 4: Collect 100 Tweets by Hashtag/Keyword

1. Use the search endpoint via rtweet::search_tweets() to collect tweets matching a hashtag or keyword. Specify n = 100 (or slightly higher to account for potential duplicates or deletions) and include options to remove retweets if desired (rtweet supports the standard search and newer API endpoints depending on access level) (Kearney, 2019).

2. Be mindful of rate limits: standard search endpoints and streaming endpoints differ in limits and behavior. For reproducible classroom projects, staying within n = 100 will usually avoid exceeding common rate limits (Morstatter et al., 2013).

Example conceptual code (do not insert keys):

# Authenticate (reads from .Renviron)

library(rtweet)


token 

Search for 100 tweets with a hashtag


hashtag_tweets

Step 5: Collect 100 Tweets from a User Profile

1. Use rtweet::get_timeline() to retrieve recent tweets from a specific user. Set n = 100 to get up to 100 most recent tweets from the account timeline (Kearney, 2019).

2. Check for protected accounts—timelines for protected users are inaccessible unless you have permission. Also be aware of user suspension or deletion which can reduce the returned count (Bruns & Stieglitz, 2013).

Example conceptual code:

# Get 100 tweets from a user timeline

user_tweets

Step 6: Validate and Save Collected Data

1. After collection, verify the returned data frame contains the expected number of rows and inspect key fields such as text, created_at, screen_name, id_str, and source.

2. Export results in both R-friendly formats (RDS) and interoperable formats (CSV) so the RStudio file remains a clear companion to the word/PDF file describing setup steps:

readr::write_csv(hashtag_tweets, "hashtag_tweets.csv")

readr::write_csv(user_tweets, "user_tweets.csv")

Considerations and Common Pitfalls

1. Sampling bias: The Twitter Search API and streaming APIs have different sampling characteristics; research shows discrepancies between API samples and the full firehose (Morstatter et al., 2013). For classroom tasks (n = 100), the effect is typically small, but it matters for research claims.

2. Rate limits and access tiers: If you need more than standard limits, consider applying for elevated or academic access. Respect Twitter’s terms of service and rate limit headers to avoid automated blocking (Twitter Developer, 2020).

3. Ethics and privacy: Remove or anonymize sensitive user data if you plan to publish. Cite sources and obtain permissions where required (González-Bailón et al., 2013).

Deliverables Checklist

  • One word-processing/PDF document detailing the developer account setup, how credentials were stored securely, and step-by-step instructions used to authenticate and collect tweets.
  • An RStudio project file (.Rproj) and a script (.R) that demonstrates authentication (without hard-coded keys) and the two collection steps (search_tweets and get_timeline), plus exported CSVs or RDS files containing the 200 tweets.

Conclusion

Following these steps will produce a repeatable, documented dataset of 200 tweets (100 by hashtag/keyword and 100 by user timeline) while following security and ethical best practices. The rtweet package simplifies OAuth and data collection; documenting the process in a PDF and providing the RStudio project ensures reproducibility (Kearney, 2019; Silge & Robinson, 2017).

References

  • Kearney, M. W. (2019). rtweet: Collecting Twitter data. CRAN. https://cran.r-project.org/package=rtweet (Kearney, 2019)
  • Twitter Developer. (2020). Twitter Developer Platform Documentation. https://developer.twitter.com/en/docs (Twitter Developer, 2020)
  • Wickham, H., & RStudio. (2019). httr: Tools for Working with URLs and HTTP. CRAN. https://cran.r-project.org/package=httr (Wickham, 2019)
  • Silge, J., & Robinson, D. (2017). Text Mining with R: A Tidy Approach. O’Reilly Media. https://www.tidytextmining.com/ (Silge & Robinson, 2017)
  • Morstatter, F., Pfeffer, J., Liu, H., & Carley, K. M. (2013). Is the sample good enough? Comparing data from Twitter’s streaming API with Twitter’s firehose. ICWSM. https://www.aaai.org/ocs/index.php/ICWSM/ICWSM13/paper/view/6233 (Morstatter et al., 2013)
  • Tumasjan, A., Sprenger, T. O., Sandner, P. G., & Welpe, I. M. (2010). Predicting elections with Twitter: What 140 characters reveal about political sentiment. ICWSM. https://www.aaai.org/ocs/index.php/ICWSM/ICWSM10/paper/view/1536 (Tumasjan et al., 2010)
  • González-Bailón, S., Borge-Holthoefer, J., & Moreno, Y. (2013). Broadcasters and hidden influentials in online protest diffusion. American Behavioral Scientist, 57(7), 943–965. https://doi.org/10.1177/0002764213479371 (González-Bailón et al., 2013)
  • Bruns, A., & Stieglitz, S. (2013). Toward more systematic Twitter research: Introducing the Twitter data collection and analysis framework. Journal of Social Media Studies, 1(1). https://eprints.qut.edu.au/66878/ (Bruns & Stieglitz, 2013)
  • R Core Team. (2020). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.R-project.org/ (R Core Team, 2020)
  • Silge, J. (2019). Practical examples and vignettes for text mining and data collection using R and rtweet. https://juliasilge.com/blog/ (Silge, 2019)

Related Assignments