This can include scraping raw

Discuss my database trends and their role in business.
Post Reply
asimj1
Posts: 418
Joined: Tue Jan 07, 2025 4:51 am

This can include scraping raw

Post by asimj1 »

For example, I completed a big piece of work over the summer which involved building a prototype Reproducible Analytical Pipeline (RAP) using R and Github. A RAP is a computer program which automates the different stages which are involved in undertaking a piece of data analysis. data off a website, cleaning and rcs data malaysia wrangling it into a different format, and then extracting insights from it by creating visualisations and summary statistics, which will then be automatically updated each time the program runs.

The key thing about this piece of work was that, in addition to producing the analysis itself, an important goal of that project was to be able to have a working codebase hosted on Github for building RAPs which we could share with other analysts who wanted to implement a RAP using R themselves.

Given that I now work almost exclusively in both of my jobs with open-source software tools which are based on work which thousands of other developers have contributed for free, I think that publishing code which other users may end up using in their own projects is an important new way of generating impact from the things I do with data. Others agree, including the Social Metrics Commission.
Post Reply