More context:
I was a 60% Data Engineer and 40% back eng engineer.
The job ads now ask for cloud computing, I did on-perm with similar components but built in house.
Step Functions
Glue
Athena
S3
RDS
Databrick GCP
Snowflake
I am looking at ways to 'modernize' 'cloudifly' myself
This is chatgpt suggestions:
Step 3: Your FIRST project (this is the breakthrough)
Project: “Cloud Data Pipeline” (must-do)
What you build:
Pull financial data (API)
Store in S3
Process with Python
Schedule with Airflow
Load into warehouse (Redshift or PostgreSQL)
Architecture (simple)
API → Python → S3 → Airflow → DB → (optional dashboard)
Upskilling is showing I commit to learning and improving myself, and getting a shot in the interview by not 'lying' to list the skills on my CV. I had an interview the other day and the take home assessment was actually fun for me. Though the vibe of the company didn't really click so I kinda show in the interview. Ouch... another story.
I actually think if given some ramp up time, I would learn the skills while on the job. Because decade in tech meant that I have done that multiple times.
I am a super social anxious so the first 2 steps felt a bit daunting. I moved (family reason - caregiving) so my 'best work buddies' are in another city/country. I did reach out to 2 or 3 but no leads there. I am from a company that people 'used' to stay there a very long time.
I joined a local mentoring group in STEAM and expanding my network.
For step 3, I put in
Side Project:
Python
Coding Agent
AWS cloud platform
I didn't 'code' or read any CS related for almost a year. I agreed that now I didn't need more 'input' course material, but rather a 'learn and output' way to rebound.
Since last October, I have been using github co-pilot(because its free, unemployed) to write little python project helper for my finance admin and other daily stuff. Initially I had to 'plan' with AI, broke tasks into smaller tasks and modified a few things. I haven't 'written' a single line of code since January as the model is so powerful they can step thru the changes and debugging. Granted this is a very small project.
I aslo think contributions to the opensource projects would be helpful, for the sake of mental health and having 'real' practice.
I am also thinking about might be start with some opensource project that I have used, e.g. like pyarrows, pandas, jupyter for python, and spark for scala. However, I think I am actually more interested in building a system together, rather than writing 'libraries'.
Do you know how I can find one? I had tried searching via google but its not effective. I guess I don't know how.
I have also tried to find 'volunteer jobs' but not very successful. Again, I think I might not know 'where' to look.
I think my mind is still all over the place after the burnout so would need some brain power from the community.
I have never worked on compiler. More on application level. I did have a lot of benchmarking experience on architecture choice so might be that is a skill I could contribute.
The job ads now ask for cloud computing, I did on-perm with similar components but built in house.
Step Functions Glue Athena S3 RDS
Databrick GCP Snowflake
I am looking at ways to 'modernize' 'cloudifly' myself
This is chatgpt suggestions: Step 3: Your FIRST project (this is the breakthrough) Project: “Cloud Data Pipeline” (must-do) What you build: Pull financial data (API) Store in S3 Process with Python Schedule with Airflow Load into warehouse (Redshift or PostgreSQL) Architecture (simple) API → Python → S3 → Airflow → DB → (optional dashboard)
reply