DuyNK
  • Home
  • Work
  • Sharings
    • Knowledge
    • Concepts
    • Notes
    • Experiments
    • Glossary
    • Resources
    • Slides
    • Templates
    • Datasets
    • Tools
  • Contact
Knowledge
  • Concepts
    • Artificial Intelligence
      • Machine Learning
        • Data
          • Structured vs Unstructured
          • Tabular Data
          • Time Series
        • Evaluation
          • Bias-Variance Tradeoff
          • Cross-Validation
          • Overfitting / Underfitting
        • Features
          • Dimensionality Reduction
          • Feature Engineering
          • Feature Selection
          • Hand-crafted Features
        • Learning Paradigms
          • Semi-Supervised Learning
          • Supervised Learning
          • Unsupervised Learning
          • Weakly Supervised Learning
        • Models
          • Decision Tree
          • Gradient Boosting
          • k-NN
          • Linear Regression
          • Logistic Regression
          • Random Forest
          • SVM
      • Deep Learning
        • Architectures
          • AlexNet
          • CNN
          • DenseNet
          • Diffusion Models
          • EfficientNet
          • GAN
          • GoogLeNet / Inception
          • LeNet-5
          • LSTM / GRU
          • MLP
          • MobileNet
          • PatchCore
          • ResNet
          • RNN
          • Swin Transformer
          • Transformer
          • U-Net
          • VGGNet
          • Vision Transformer (ViT)
          • YOLO
        • Language Models
          • Context Windows
          • Context Rot
        • Neural Network Basics
          • Activation Function
          • Backpropagation
          • Batch Normalization
          • Dropout
          • Loss Function
          • Neuron
          • Weights & Bias
        • Tasks
          • Anomaly Detection
          • Classification
          • Detection
          • Segmentation
        • Training
          • Batch Size
          • Initialization
          • Learning Rate
          • Optimizers
          • Regularization
        • Vision-Specific Concepts
          • Attention
          • Convolution
          • Data Augmentation
          • Kernel
          • Pooling
          • Receptive Field
      • Reinforcement Learning
    • Cross-Domain Concepts
    • Mechanical Engineering
      • Bending Moment & Shear Force
      • Column Buckling
      • Factor of Safety
      • Failure Theories
      • Fatigue & S-N Curves
      • GD&T
      • Heat Transfer
      • Hooke's Law
      • Material Properties
      • Stress & Strain
      • Stress-Strain Curves
      • Types of Loads
    • Project Management
      • Overview
      • Mindset
        • Proactive
        • Ownership
        • Value-Driven
      • Principles
        • Holistic View
        • Embed Quality
        • Accountable Leadership
        • Empowered Culture
        • Focus on Value
        • Sustainability
      • Performance Domains
        • Governance
          • Governance Processes
            • Initiate Project or Phase
            • Integrate and Align Project Plans
            • Plan Sourcing Strategy
            • Manage Project Execution
            • Manage Quality Assurance
            • Manage Project Knowledge
            • Monitor and Control Project Performance
            • Assess and Implement Changes
            • Close Project or Phase
          • Governance Model
          • Tailoring & Effectiveness
        • Scope
        • Schedule
        • Finance
        • Stakeholders
        • Resources
        • Risk
      • Focus Areas
        • Initiating
        • Planning
        • Executing
        • Monitoring and Controlling
        • Closing
      • Concepts
        • Value Delivery
        • Tailoring
        • Project Life Cycle
        • Outputs vs Outcomes
        • Predictive
        • Adaptive
        • Hybrid
        • Benefits
      • Methods
        • Work Breakdown Structure
        • Critical Path
        • Earned Value
        • Project Financial Evaluation
        • RACI
        • Risk Matrix
        • Monte Carlo Simulation
      • Artifacts
        • Project Charter
        • Business Case
          • Business Case Guide
          • Water Treatment Capacity Upgrade
          • Customer Support Platform
          • Automated Visual Inspection
          • Remote Patient Monitoring
          • Solar and Battery System
        • Risk Register
        • Issue Log
        • Stakeholder Register
        • Lessons Learned
      • Scenarios
        • Scope Creep
        • Stakeholder Conflict
        • Schedule Delay
        • Supplier Risk
        • Quality Issue
    • Software Engineering
    • Systems & Control
  • Experiments
  • Glossary
    • AI
    • ME
    • Metrics
    • PM
    • Software Engineering
  • Notes
    • Agent Architecture
      • Hermes AI Team Architecture
    • AI Fundamentals
      • Artificial Intelligence (AI)
    • Tools
      • Syncthing — My Simple Local-First File Sync Setup
      • Windows Terminal: mac6 SSH Launcher
      • Windows Terminal Pane Management
    • Vibe Coding vs Agentic Coding
Knowledge
  • Concepts
    • Artificial Intelligence
      • Machine Learning
        • Data
          • Structured vs Unstructured
          • Tabular Data
          • Time Series
        • Evaluation
          • Bias-Variance Tradeoff
          • Cross-Validation
          • Overfitting / Underfitting
        • Features
          • Dimensionality Reduction
          • Feature Engineering
          • Feature Selection
          • Hand-crafted Features
        • Learning Paradigms
          • Semi-Supervised Learning
          • Supervised Learning
          • Unsupervised Learning
          • Weakly Supervised Learning
        • Models
          • Decision Tree
          • Gradient Boosting
          • k-NN
          • Linear Regression
          • Logistic Regression
          • Random Forest
          • SVM
      • Deep Learning
        • Architectures
          • AlexNet
          • CNN
          • DenseNet
          • Diffusion Models
          • EfficientNet
          • GAN
          • GoogLeNet / Inception
          • LeNet-5
          • LSTM / GRU
          • MLP
          • MobileNet
          • PatchCore
          • ResNet
          • RNN
          • Swin Transformer
          • Transformer
          • U-Net
          • VGGNet
          • Vision Transformer (ViT)
          • YOLO
        • Language Models
          • Context Windows
          • Context Rot
        • Neural Network Basics
          • Activation Function
          • Backpropagation
          • Batch Normalization
          • Dropout
          • Loss Function
          • Neuron
          • Weights & Bias
        • Tasks
          • Anomaly Detection
          • Classification
          • Detection
          • Segmentation
        • Training
          • Batch Size
          • Initialization
          • Learning Rate
          • Optimizers
          • Regularization
        • Vision-Specific Concepts
          • Attention
          • Convolution
          • Data Augmentation
          • Kernel
          • Pooling
          • Receptive Field
      • Reinforcement Learning
    • Cross-Domain Concepts
    • Mechanical Engineering
      • Bending Moment & Shear Force
      • Column Buckling
      • Factor of Safety
      • Failure Theories
      • Fatigue & S-N Curves
      • GD&T
      • Heat Transfer
      • Hooke's Law
      • Material Properties
      • Stress & Strain
      • Stress-Strain Curves
      • Types of Loads
    • Project Management
      • Overview
      • Mindset
        • Proactive
        • Ownership
        • Value-Driven
      • Principles
        • Holistic View
        • Embed Quality
        • Accountable Leadership
        • Empowered Culture
        • Focus on Value
        • Sustainability
      • Performance Domains
        • Governance
          • Governance Processes
            • Initiate Project or Phase
            • Integrate and Align Project Plans
            • Plan Sourcing Strategy
            • Manage Project Execution
            • Manage Quality Assurance
            • Manage Project Knowledge
            • Monitor and Control Project Performance
            • Assess and Implement Changes
            • Close Project or Phase
          • Governance Model
          • Tailoring & Effectiveness
        • Scope
        • Schedule
        • Finance
        • Stakeholders
        • Resources
        • Risk
      • Focus Areas
        • Initiating
        • Planning
        • Executing
        • Monitoring and Controlling
        • Closing
      • Concepts
        • Value Delivery
        • Tailoring
        • Project Life Cycle
        • Outputs vs Outcomes
        • Predictive
        • Adaptive
        • Hybrid
        • Benefits
      • Methods
        • Work Breakdown Structure
        • Critical Path
        • Earned Value
        • Project Financial Evaluation
        • RACI
        • Risk Matrix
        • Monte Carlo Simulation
      • Artifacts
        • Project Charter
        • Business Case
          • Business Case Guide
          • Water Treatment Capacity Upgrade
          • Customer Support Platform
          • Automated Visual Inspection
          • Remote Patient Monitoring
          • Solar and Battery System
        • Risk Register
        • Issue Log
        • Stakeholder Register
        • Lessons Learned
      • Scenarios
        • Scope Creep
        • Stakeholder Conflict
        • Schedule Delay
        • Supplier Risk
        • Quality Issue
    • Software Engineering
    • Systems & Control
  • Experiments
  • Glossary
    • AI
    • ME
    • Metrics
    • PM
    • Software Engineering
  • Notes
    • Agent Architecture
      • Hermes AI Team Architecture
    • AI Fundamentals
      • Artificial Intelligence (AI)
    • Tools
      • Syncthing — My Simple Local-First File Sync Setup
      • Windows Terminal: mac6 SSH Launcher
      • Windows Terminal Pane Management
    • Vibe Coding vs Agentic Coding
  1. Knowledge
  2. /Concepts
  3. /Artificial Intelligence
  4. /Deep Learning
  5. /Neural Network Basics

Neural Network Basics

The fundamental building blocks that every neural network is made of. Here you will find explanations of neurons, layers, activation functions, loss functions, and the core mechanics of forward and backward propagation.

  • Activation FunctionNon-linear transformations in networks
  • BackpropagationGradient computation through the network
  • Batch NormalizationBatch Normalization - Normalizing layer inputs to stabilize and accelerate training
  • DropoutDropout - Regularization by randomly deactivating neurons during training
  • Loss FunctionMeasuring model prediction error
  • NeuronThe basic computational unit
  • Weights & BiasLearnable parameters in neural networks
← Back to Knowledge
EmailGitHubLinkedIn

© 2026 DuyNK. Built with Next.js.