My Time so far in The Coding Schools Research Program

-

I've been in the Data Science Research Program hosted by The Coding School, where I've learned a ton about the data science process. 


Here's the process in case you're interested:


1. Data Cleaning: removed null and missing values, filtering price outliers that skewed the data.


2. Exploratory Data Analysis (EDA): a viewing phase where you look for cool insights in the data that you can explore through functions, plots, and further visualization techniques. 


3. Statistical Analysis: Developing a null and alternative hypothesis, basically just a claim through the data that does not correlate (Ex, People who study more will not have a significantly higher grade on the final exam) and one that correlates (Ex, People who study more will have a significantly higher grade on the final exam). We can test for significance using a variety of tests, including T-tests, ANOVA, and Tukey's HSD. This will help you decide whether to approve/reject the null hypothesis based on the p-values they produce, but I won't go into too much detail because it is pretty complicated.


4. ML modeling: Once you've accepted/rejected the null hypothesis, you can see if the predictability of your data vs your desired target variable. For example, using the earlier claim, if you found that studying more improved final exam scores, you can use a linear regression model to see how a linear function can predict scores based solely on study duration. You can train the model on 80% of the data, allowing it to see how studying affects the score, then test it on the remaining 20%.


5. Evaluating your model: This step can help you understand the effectiveness of your model. You can do this using tools like R², mean squared error (MSE), and mean absolute error (MAE), along with a scatter plot comparing your model to the actual data. 


Anyways, going back to the program, although it is rigorous, I find it incredibly insightful because I understand the importance of what I am doing. Learning the fundamentals of matplotlib, pandas, seaborn, and sklearn helps me visualize what life is like as a data scientist. Through the program, I've also had the opportunity to speak with established experts, including those from Google, Harvard, Apple, and even the US Navy. These guest speakers provide real-world implications of data science, and the diverse range of speakers helps me understand how data can benefit any business, person, or country.


We have now started the research section of the program, which runs for the final 3 weeks. I have a mentor who provides advice on my project and argument, along with a TA for specific debugging issues. I can't wait to share my results, but I first have to finalize my paper and present my results. 

Comments

Post a Comment

Popular posts from this blog

Do the 3Cs (Cut, Color, Clarity) Really Impact the Overall Pricing of a Diamond?

-
Hello everyone!  My experience in the Coding School National High School Research Program has concluded, and I've completed my research on the impact of cut, color, clarity, and carat weight on a diamond's overall value. I'd also like to extend a special thank you to The Coding School and my mentor, Sujay Shetty, for all the fantastic guidance; your help made this possible. Why I Did This Project  I wanted to explore product pricing and how items could be priced based on their key characteristics. While I didn't immediately think diamonds would be my topic of choice, it became fascinating once I delved deep into the subject. My dataset, sourced from Kaggle, has over 50,000 entries, making EDA incredibly worthwhile for uncovering trends. As I delved deeper into the data, my curiosity only grew, and h ere's  what I discovered .  What I Found Using methods described in my paper, I found that around 90% of level changes for the 3Cs significantly affected price; however,...
Read more

My Response to a TED Talk by Mallory Freeman

-
Hello everyone! I recently watched a great Ted Talk by Mallory Freeman titled "Your company's data could help end world hunger." In it, she describes her experiences in optimizing trade routes to reduce food costs and, consequently, feed more people. How did she do this? She used data to build a model that identifies the optimal route for food. Freeman's accomplishments are built on data. Without it, she wouldn't be able to make a model to feed thousands more individuals. She is a firm believer that "information visibility drives efficiency." So she proposes three advancements that would benefit humanitarian organizations, tailored to how companies share and manage their internal data and analytics teams. Companies should: Share data with humanitarian organizations Lend their data scientists/decision scientists who can clean and model data Share their new innovations/technologies to allow for more data collection Freeman closes her talk discu...
Read more

Brookline Business Research | Part 2: The Data

-
Image
Hello people! Recently, I visited businesses in Brookline to better understand how policies are affecting them. I asked a wide range of local shops to better represent the diverse Brookline landscape, including restaurants (casual and sit-down), cafes, specialty stores, and supermarkets. While I cannot provide specific information about some of the businesses in my sample due to a desire to remain anonymous, what I have below is data worth looking into more deeply. --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- The Data --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Now What? I will delve deeper into my interpretation of this data in a future blog post. Analyzing this could help me understand the disparit...
Read more