Carisa's Dev Journey Notes

A collection of notes I have taken

Table of Contents

Document Key

Real World Projects

Project 1: Large-Dataset-Generation

Breakdown of Project 1

Project 2: Data-Analysis-and-Interpretation

Breakdown of Project 2

Programming Languages

Programming language: Bun

Programming language: LangChain

Programming language: NumPy

Programming language: Prometheus

Programming language: PyTorch

Programming language: Sentry

Programming language: Svelte

Programming language: SvelteKit

Programming language: Threlte

Programming language: Whisper

Real World Guidelines

Table of Contents Key

Table of Contents

Breakdown Table of Contents

Wiki Home Page Key

Wiki Home Page

Breakdown Wiki Home Page

Personal Project Key

Personal Project Guidelines

Project Initialization

Code Management

Testing and Debugging

Code Review and Refactoring

Security and Best Practices

Documentation

Project Completion

Step-by-step Personal Project Guidelines

Step 1: Project Initialization

Step 2: Development 26

Step 3: Testing and Debugging

Step 4: Security and Best Practices

Step 5: Documentation

Step 6: Project Completion

Document Key

Highlight Colors

• Pink: Real World Projects

• Light Purple: Programming Languages

• Light Blue: Real World Guidelines

• Light Yellow: Title (Main Topic)

• Teal: Subtitle (Tailored to Topic)

• Red: Date Started

• Gray: Date Finished

Real World Projects

Project 1: Large-Dataset-Generation

Project Description: Generates large dataset from News API

Programming Languages: JavaScript, Node.js, and Axios

Programming Tools: News API, Visual Studio Code, Git, and GitHub

Start Date: August 4, 2024

End Date: August 6, 2024

Breakdown of Project 1

Strengths:

• Clear documentation: README well structured, good explanation of project’s purpose, setup, and usage

• Specific use: Project has a specific use (generating large dataset from News API)

• Technologies Used: Project uses popular and widely used technologies (Node.js, JavaScript, and Axios)

Weaknesses:

• Limited customization: Project is only designed for a specific case (fetching articles about Elon Musk). Adjust to adapt to other use cases.

• Dependence on News API: Project relies heavily on the News API which creates limitations (rate limit, data quality, possible change to API endpoint)

• Data storage: Project stores data in text file, not suitable for large-scale data storage/analysis

Potential for real-world/company use:

• Media monitoring: Companies can use to monitor news articles about their brand, competitors, or industry trends

• Market research: Researchers can use to gather data on market trends, customer opinions, or product reviews

• Data analysis: Data analyst can use as starting point for more complex analysis tasks (sentiment, topic modeling)

Improvements for real-world/company use:

• Add customization options: Allow users to specific their own API (endpoints, query parameters, data storage location)

• Improve data storage: Consider using more robust data storage location (database, data warehousing)

• Enhance error handling: Add more robust error handling to make sure project can handle API errors, network issues, etc

Project 2: Data-Analysis-and-Interpretation

Project Description: Data analysis and interpretation project that uses R to analyze and evaluate news article data

Programming Languages: R

Programming Tools: News API, Replit, and GitHub

Start Date: August 4, 2024

End Date: August 8, 2024

Breakdown of Project 2

Strengths:

• Clear methodology: Project follows clear defined methodology (makes it easy to understand, replicate)

• Organized code: Code is organized (tasks separated, clear sections)

• Effective data visuals: Bar plot, effective way to show findings (clear title, labels- easy to understand)

• Data exploration: Project has data exploration steps (view structure of data frame, checking missing values)

Weaknesses:

• Limited data validation: Project does not have data validation steps (leads to - errors, inconsistencies in data)

• Limited data cleaning: Project does not have data cleaning steps (leads to - errors, inconsistencies in data)

• Limited data transformation: Project does not have data transformation steps (difficult to analyze data)

• Limited reproducibility: Project does not have any steps to make results more reproducible (setting random seed, robust method generating plot)

Potential for real-world/company use:

• Analyzing Customer Feedback: Company use project’s methodology analyze (customer feedback, identify trends, and patterns in satisfaction)

• Monitoring Website Traffic: Analyze website traffic data, identify trends, and patterns of user behavior

• Analyzing Sales Data: Analyze sales data, identify trends, and patterns in sales performance

Improvements for real-world/company use:

• Data Validation Steps: Add validation steps, ensure data is in expected format (no errors in data)

• Data Cleaning Steps: Include data cleaning steps to handle missing values, inconsistent data

• Methodology: Include advanced data analysis and visualization techniques

• Results Reproducible: Add steps for results more reproducible

Programming Languages

Programming language: Bun

Programming definition: All-in-one JavaScript runtime and toolkit designed for speed and efficiency

Examples of use:

• Run JavaScript/TypeScript applications: bun run index.ts

• Installing packages: bun install react

• Bundling code: bun build ./src/index.ts --outdir ./dist

• Run tests: bun test

• Starting a development server: bun --hot run server.ts

Features:

• Fast, JavaScript and TypeScript

• Built-in package manager compatible with npm

• Bundler for creating optimized production builds

• Test runner for executing unit and integration tests

• Hot reloading for quick development iterations

Programming language: LangChain

Programming definition: Framework for developing applications powered by language models

Examples of use:

• Creating a chatbot (conversational AI using Large Language Models(LLMs))

• Document analysis: Extract information from large text datasets

• Question-answering systems: Build systems that can answer queries based on given context

• Automated content generation: Create tools for writing articles or summarizing text

Features:

• Integrates with various LLMs (e.g., GPT-3, GPT-4)

• Tools for prompt engineering and management

• Offers memory systems for context retention in conversations

• Enables creation of AI agents that can use external tools

Programming language: NumPy

Programming definition: fundamental package for scientific computing in Python

Examples of use:

• Data analysis: Perform statistical operations on large datasets

• Image processing: Manipulate and analyze image data as multi-dimensional arrays

• Machine learning: Prepare and process data for ML algorithms

• Financial modeling: Conduct complex calculations on financial data

Features:

• Efficient multi-dimensional array object

• Functionality for operating on arrays of different shapes

• Tools for integrating C/C++ and Fortran code

• Linear algebra operations, Fourier transform, and random number capabilities

Programming language: Prometheus

Programming definition: Open-source monitoring and alerting toolkit

Examples of use:

• Server monitoring: Track CPU, memory, and disk usage of servers

• Application performance monitoring: Measure response times and error rates

• Container schedule monitoring: Monitor Kubernetes clusters

• Network monitoring: Track network traffic and connectivity issues

Features:

• Multidimensional data model with time series data identified by metric name and key/value pairs

• PromQL, a flexible query language for data analysis

• No reliance on distributed storage (single server nodes are autonomous)

• Push gateway for supporting short-lived jobs

• Service discovery or static configuration for scraping targets

Programming language: PyTorch

Programming definition: Open-source machine learning library (developed by Facebook's AI Research lab)

Examples of use:

• Deep learning research: Implement and experiment with neural network architectures

• Computer vision: Develop image classification, object detection, and segmentation models

• Natural language processing: Create language models, machine translation systems, and chatbots

• Reinforcement learning: Learn to make decisions in complex environments

Features:

• Dynamic computational graphs for flexible model building

• Integration with Python tools

• GPU acceleration for fast model training

• Rich ecosystem of tools and libraries for various AI tasks

• TorchScript for model deployment in production environments

Programming language: Sentry

Programming definition: Error monitoring platform that helps identify, track, and resolve issues in applications

Examples of use:

• Web application monitoring: Track errors in frontend and backend code

• Mobile app crash reporting: Identify and diagnose issues in iOS and Android apps

• Performance monitoring: Analyze application performance and identify bottlenecks

• Release tracking: Associate errors with specific software releases

Features:

• Real-time error tracking and reporting

• Detailed stack traces and error context

• Integration with development tools and platforms

• Customizable alerts and notifications

• Support for multiple programming languages and frameworks

Programming language: Svelte

Programming definition: Modern JavaScript framework for building user interfaces

Examples of use:

• Single-page applications: Build interactive web applications

• Component libraries: Create reusable UI components

• Data visualization: Develop interactive charts and graphs

• Progressive web apps: Build responsive, offline-capable web applications

Features:

• Compile-time framework with no virtual DOM

• Reactive declarations for easy state management

• Scoped styling to prevent CSS conflicts

• Built-in animations and transitions

• Small bundle sizes for faster loading times

Programming language: SvelteKit

Programming definition: Application framework built on top of Svelte for creating full-stack web applications

Examples of use:

• E-commerce websites: Build online stores

• Blogs and content management systems: Create dynamic content websites

• Web applications with authentication: Develop secure multi-user applications

• Server-side rendered (SSR) applications: Improve initial page load times and SEO

Features:

• File-based routing for project structure

• Server-side rendering

• API route creation for backend functionality

• Adapter system for easy deployment to various platforms

• Code splitting for optimized loading times

Programming language: Threlte

Programming definition: Library, 3D graphics capabilities to Svelte applications using Three.js

Examples of use:

• 3D product visualizations: Create interactive 3D models for e-commerce

• Data visualizations: Develop 3D charts and graphs

• Game development: Build browser-based 3D games

• Virtual tours: Create immersive 3D environments for real estate or museums

Features:

• Declarative syntax for creating 3D scenes in Svelte

• Reactive updates to 3D objects based on state changes

• Integration with Svelte component system

• Optimized performance for rendering complex 3D scenes

• Easy animation and interaction implementation

Programming language: Whisper

Programming definition: Automatic speech recognition (ASR) system developed (OpenAI)

Examples of use:

• Transcription services: Convert audio files or live speech to text

• Closed captioning: Generate subtitles for videos automatically

• Voice assistants: Implement speech recognition for voice commands

• Language learning tools: Create applications for practicing pronunciation

Features:

• Multilingual support for transcribing and translating many languages

• Robust performance across various accents and audio qualities

• Zero-shot learning capabilities for adapting to new tasks

• Open-source availability for research and development

• Handle technical language and domain-specific vocabulary

Real World Guidelines

Structured template of table of contents for GitHub Documentation

Table of Contents Key

Highlight Colors

• Mint: H1

• Medium Purple: H2

• Light Red: H3

• Light Pink: H4

• Medium Blue: Horizontal Break

Table of Contents

Project Title

Project Description

Table of Contents

• Project Overview

  • Project Description

  • Project Goals

  • Key Features

• Getting Started

  • Prerequisites

  • Setup Instructions

  • Installation

• Features

  • Examples

• Technical Requirements

  • Hardware Requirements

  • Software Requirements

  • Network Requirements

• Technical Details

  • Architecture

  • Project Structure

  • Packages (If used)

• Security (If used)

  • Security Overview

  • Security Best Practices

• Wiki

  • Wiki Overview

• Contributing

  • Contributing Guidelines

  • Code of Conduct

• License and Copyright

  • License

  • Attribution

• Contact and Support

  • Contact Information

    • About Developer

  • Support Resources

    • FAQs

    • Glossary

    • Documentation

Breakdown Table of Contents

Project Title

Project Description

• Brief summary 1-2 sentences

***

Table of Contents

***

Project Overview

***

Project Description

• More detailed explanation of project 2-3 paragraphs

Project Goals

• List project goals

Key Features

• Key Features of the project

Getting Started

***

Prerequisites

• List necessary prerequisites for using project, what you need to get started

Setup Instructions

• List setup configuration requirements to use project

Installation

• Step-by-step instructions for installing project

Features

***

• List project features, include notable functions, integration, or tools

• (Use this section to highlight project capabilities and benefits)

Examples

• Include examples or use cases to illustrate project functionality

Technical Requirements

***

• (Use this section to help readers understand project, technical dependencies, and limitations)

Hardware Requirements

• List hardware requirements (RAM, CPU, etc)

Software Requirements

• List software requirements (Visual Studio Code, libraries, etc)

Network Requirements

• List network requirements (internet connectivity, etc)

Technical Details

***

Architecture

• Describe project architecture (Visuals- Diagrams, design process)

Project Structure

• List and describe file structure of project

Packages (If used)

• List and describe packages used in the project

Security (If used)

***

Security Overview

• Describe overview of project’s security features and considerations

Security Best Practices

• List and describe best security practices for using the project

Wiki

***

Wiki Overview

• Brief overview of wiki and purpose

Contributing

***

Contributing Guidelines

• Describe guidelines for contributing to the project (how to join the project)

Code of Conduct

• Describe code of conduct for people who want to join

License and Copyright

***

License

• Specify project’s license and any relevant copyright information

Attribution

• Explain any credit or acknowledgments (third-parties libraries, frameworks, etc) used in the project. Explain any sources or inspirations (research, code, borrowed or adapted) used in the project

Contact and Support

***

Contact Information

• Provide contact information (email, supporting channels-youtube)

About Developer

• Short bio of me (something not from my resume, GitHub profile)

Support Resources

FAQs

• List question and answers (general questions about project- information that is not explained in README)

Glossary

• List key terms or concepts used in the project

Documentation

• List and describe sources used (explain the why I went that route)

Structured template of wiki for GitHub Documentation

Wiki Home Page Key

Highlight Colors

• Mint: H1

• Medium Purple: H2

• Light Red: H3

• Light Pink: H4

• Medium Blue: Horizontal Break

Wiki Home Page

Wiki Title

Wiki Overview

Table of Contents

• Introduction

• Tutorials

  • Title of tutorial

  • Tutorial instruction steps

• Methodologies

  • Name of methodology

    • Overview

    • Benefits

    • Challenges

• Approaches

  • Name of approach

    • Overview

    • Benefits

    • Challenges

• Resources

  • Name of resource

    • Link (name)

    • Description

  • Glossary

  • References

• FAQs

Breakdown Wiki Home Page

Wiki Title

Wiki Overview

• Brief overview of wiki’s purpose 1-2 sentences

***

Table of Contents

Introduction

***

• More detailed explanation of wiki’s purpose 1-2 paragraphs

Tutorials

***

Title of tutorial

• Name of the tutorial

• (explain purpose of tutorial- reason for having it, why is it important)

Tutorial instruction steps

• Detailed instructions (step 1, 2, etc)

Methodologies

***

Name of methodology

• Name of methodology

Overview

• Detailed explanation of methodology (include explanation of the why it’s being used)

Benefits

• List benefits of using methodology

Challenges

• List any challenges experienced with methodology

Approaches

***

Name of approach

• Name of approach

Overview

• Detailed explanation of approach (include explanation of the why it’s being used)

Benefits

• List benefits of using approach

Challenges

• List any challenges experienced with approach

Resources

***

Name of resource

• Name of resource

Link

• Link to the resource

Description

• Explanation of the resource (why it was used)

Glossary

• List key terms or concepts used in the project

References

• Explain any sources used in wiki

FAQs

***

• List question and answers (general questions about wiki- information that is not explained in wiki

Structured real-world guidelines for any project

Personal Project Key

Highlight Colors

• Light Yellow: Title (Main Topic)

• Teal: Subtitle (Tailored to Topic)

• Medium Yellow: Word Definitions

• Medium Green: Create Separate Template

Personal Project Guidelines

Project Initialization

• Define project goals or objectives

  • Clearly explain what you want to achieve by building the project (it’s purpose)

• Conduct feasibility assessment (need to create)

  • Assess resources, time, and skills to do the project

• Create a project plan

  • Create a plain (milestones and deadlines)

• Setup repository

  • Create project repository GitHub

Code Management

• Use version control system

  • Use a version control system, Git

• Follow a coding standard (need to create coding standard)

  • Establish coding standard (ensure consistency and readability across code base)

  • Code base = foundation of software, includes all files needed to compile the software

• Write clean and modular code

  • Reusable modules to improve maintainability and scalability

  • Modular code = Breaking code into smaller separate pieces that do one specific thing. If one part needs fixing or changed, work on specific notes: part without breaking entire code

• Use a code editor or IDE

  • Choose correct editor or IDE for project

  • Code editor = write and edit code, simple (Ex: notepad)

  • IDE (Integrated Development Environment) = all-in-one solution, writing, testing, and managing code

Testing and Debugging

• Write unit tests

  • Create unit tests (ensure code quality and catch errors)

• Use a testing framework (create testing framework template based on language)

  • Use testing framework like Playwright (other options- Selenium, Jest, Pytest, Unittest, etc)

• Use a debugger (create debugger based on language)

  • Use a debugger like Chrome DevTools (other options- Node.js Inspector, Angular Augury, SonarLint, etc)

Code Review and Refactoring

• Regularly review code (figure out when to stop building and start review)

  • Schedule review of code to ensure code quality and identify areas for improvement

• Refactor code

  • Refactor to improve maintainability, scalability, and performance

  • Refactor code = clean up or improve the structure of the code without changing what the code does (Ex: Make code simpler to read, remove duplicate code, break big pieces into smaller and reusable parts)

• Use code analysis tools (create and figure out when to use analysis tools)

  • Use code analysis tools (never use any- Code Factor, Code Climate, PMD, etc)

• Keep code organized

  • Keep code organized (comments, documentation, and formatting)

Security and Best Practices

• Follow security best practices (need to create security practice)

  • Implement security practices, secure coding practices, secure data storage, and secure authentication

• Use security tools

  • Use security tools (never used any- Zap, Bearer, Horusec, etc) to identify and address security

• Keep dependencies up-to-date

  • Do regular updates on dependencies (ensure latest security patches and features)

  • Dependencies = External tools or libraries in project (extra features)

Documentation

• Write documentation

  • Create documentation (comments, README)

• Keep project journal

  • Track progress, lessons learned, plan future improvements

• Use documentation templates (need to create)

  • Ensure consistency across projects

Project Completion

• Define project completion criteria (need to create)

  • Create clear criteria for project completion (what constitutes a completed project)

• Create project completion checklist (need to create)

  • Create checklist to ensure all tasks are completed before completed

• Conduct final project review

  • Perform review of project, met requirements of completion

• Document lessons learned

  • Document lessons learned during project (improve future project outcomes)

Step-by-step Personal Project Guidelines

Project: purpose, what you are building

Project Goal: short explanation of project, purpose for building, “it’s the why are you building”

Project Objectives: explain, what you want to achieve with the project

Step 1: Project Initialization

• Define project goals and objectives

  • Review project goals and objectives (can it be improved from brainstorming, more detailed, tailored focused)

  • (write down project goals and objectives in project plan document)

• Conduct feasibility study

  • Research existing applications of what you are building (identify features, and functionality)

  • Determine technical feasibility of project (do you know the programming language for the project (if not, time to learn) framework, and tools)

  • Identify potential roadblocks and challenges (learning curve, any problem you can think of)

• Create project plan

  • Breakdown project into smaller tasks and milestones

  • Create a project timeline with est. completing dates

  • Identify dependencies and (API, library, etc) critical path tasks (create tests at review checkpoints)

• Setup project repository

  • Create repository, GitHub (README, gitignore) (start basics for version control, documentation)

Step 2: Development

• Code management

  • Use coding standard and follow throughout project (start version control, even if its small)

• Create frontend

  • Design wireframe (outline, mindmap, etc) and user interface

  • Use framework to start building UI (basic from wireframe “improve over time of project”)

• Create backend

  • Setup backend language (API, Node.js, Python, etc)

• Create authentication and authorization

  • Setup library for authentication Supabase (Passport, etc)

  • Setup library for authorization Supabase (Casbin, Remult, etc) or role-based access control (Casbin, OpenFGA, etc)

  • Authentication = Proving who you are (Ex: logging into email)

  • Authorization = Deciding what you're allowed to do (Ex: After logging into your email, decides if you can read, send, or delete emails.)

Step 3: Testing and Debugging

• Write unit tests

  • Use testing framework like Playwright (other options- Selenium, Jest, Pytest, Unittest, etc)

  • Test individual components and functions

• Write integration tests

  • Use testing framework (FitNesse, Citrus, etc) to test application’s functionality and user interface

• Debug code

  • Use a debugger to identify and fix issues (don’t forget console logs and print statements to debug code)

Step 4: Security and Best Practices

• Follow security best practices

  • Use HTTPS and SSL/TLS to encrypt data

  • Secure password storage library like bcrypt

  • Secure authentication and authorization

• Use security tools

  • Use a tools like ZAP identify vulnerabilities

  • Use SSL Checker test SSL/TLS configuration

• Keep dependencies up-to-date

  • Use npm or pip to manage dependencies

  • Regularly update dependencies to ensure you have the latest security patches

Step 5: Documentation

• Write documentation

  • Create a README explains how to use the application

  • Write documentation (API, frontend, back, etc)

• Use documentation templates

  • Use a template like Slite (The Good Docs Project) to create documentation and GitHub to host

Step 6: Project Completion

• Define project completion criteria

  • Determine what constitutes a completed project

  • Create a checklist of tasks to complete

• Conduct a final project review

  • Review project to ensure it meets the goals and objectives

  • Identify areas for improvement and create a plan to address them

• Document lessons learned

  • Write a post document summarizes the project

  • Identify lessons learned and areas for improvement