No description
Find a file
dlawler489 b8d0416a79 Fix CSV import rate limiting to prevent HTTP 429 errors
Frontend improvements:
- Add batch processing for expense creation (3 expenses per batch)
- Implement 1.5 second delays between batches to avoid overwhelming server
- Better progress logging and user feedback during batch processing
- Handle rate limit errors gracefully with proper error categorization

Backend improvements:
- Add specific rate limiter for expense creation endpoint (50 per minute)
- More informative error messages for rate limit violations
- Separate rate limiting for expense creation vs general API usage

This prevents the HTTP 429 'Too Many Requests' errors when importing large CSV files with many individual expense records (listing fees, ads, GST entries, etc.).
2026-05-05 19:31:46 +10:00
.github Fix GitHub Actions build errors and Node.js deprecation warnings 2026-05-05 18:37:29 +10:00
client Fix CSV import rate limiting to prevent HTTP 429 errors 2026-05-05 19:31:46 +10:00
data Organize data storage and enhance security 2026-04-20 09:50:38 +10:00
server Fix CSV import rate limiting to prevent HTTP 429 errors 2026-05-05 19:31:46 +10:00
.dockerignore Add GitHub Container Registry support and automated builds 2026-04-21 06:34:59 +10:00
.gitignore Add Docker containerization support 2026-04-20 19:54:37 +10:00
build-deploy.sh Add GitHub Container Registry support and automated builds 2026-04-21 06:34:59 +10:00
CHANGELOG.md Add comprehensive CSV product import system with templates and enhanced UI 2026-05-01 13:26:33 +10:00
CONTAINER_INTERFACE_DEPLOYMENT.md Enhanced deployment with comprehensive bug fixes and documentation 2026-04-21 13:18:09 +10:00
DEPLOYMENT_GUIDE.md Enhanced deployment with comprehensive bug fixes and documentation 2026-04-21 13:18:09 +10:00
DEPLOYMENT_STATUS.md Integrate Data Import with Expenses tab 2026-05-05 12:29:07 +10:00
docker-compose.deploy-local.yml Enhanced deployment with comprehensive bug fixes and documentation 2026-04-21 13:18:09 +10:00
docker-compose.deploy.yml Add JWT local auth with protected API routes 2026-04-22 08:11:54 +10:00
docker-compose.ghcr.yml Add GitHub Container Registry support and automated builds 2026-04-21 06:34:59 +10:00
docker-compose.simple.yml Enhanced deployment with comprehensive bug fixes and documentation 2026-04-21 13:18:09 +10:00
docker-compose.temp-deploy.yml Fix GitHub Actions to create :main tag for easier deployment 2026-05-05 15:14:02 +10:00
docker-compose.yml Add Nginx reverse proxy for production deployment 2026-04-21 06:30:44 +10:00
DOCKER_DEPLOYMENT.md Add Nginx reverse proxy for production deployment 2026-04-21 06:30:44 +10:00
Dockerfile Fix API container entrypoint path to compiled JS 2026-04-23 06:08:12 +10:00
GITHUB_CONTAINER_REGISTRY.md Add GitHub Container Registry support and automated builds 2026-04-21 06:34:59 +10:00
nginx.conf Add Nginx reverse proxy for production deployment 2026-04-21 06:30:44 +10:00
nginx.deploy.conf Enhanced deployment with comprehensive bug fixes and documentation 2026-04-21 13:18:09 +10:00
package-lock.json Initial commit: Complete Etsy Business Tracker with Profit Analysis Dashboard 2026-04-20 09:44:54 +10:00
package.json Fix profit analysis calculations to include shipping expenses and other costs 2026-05-05 13:21:00 +10:00
README.md Add comprehensive CSV product import system with templates and enhanced UI 2026-05-01 13:26:33 +10:00
README_NEW.md Enhanced deployment with comprehensive bug fixes and documentation 2026-04-21 13:18:09 +10:00
validate-deployment.sh Enhanced deployment with comprehensive bug fixes and documentation 2026-04-21 13:18:09 +10:00

🎯 Etsy Business Tracker with Advanced Profit Analysis

Status: LIVE IN PRODUCTION | Last Updated: May 1, 2026

A comprehensive business intelligence platform designed specifically for Etsy sellers to track orders, analyze profit margins, manage expenses, and gain deep insights into business performance with advanced profit analysis capabilities.

🚀 DEPLOYED & READY - Quick Access

Production Deployment Options

  1. GitHub Container Registry: docker-compose.deploy.yml - Pre-built images Fastest
  2. Local Build: docker-compose.simple.yml - Auto-building 🔧 Most Reliable
  3. Enhanced Debug: docker-compose.deploy-local.yml - Full logging 🔍 Best for troubleshooting

🌐 Access URL: http://localhost:8081 (after deployment)
📖 Full Guide: See DEPLOYMENT_GUIDE.md for step-by-step instructions

LIVE FEATURES - Currently Operational

📊 Advanced Profit Analysis Dashboard - LIVE

  • Real-time Profit Tracking: Monitor profit margins across all products and time periods
  • Cost Breakdown Analysis: Detailed breakdown of material costs, Etsy fees, shipping expenses
  • Order Drill-Down Analysis: Product-specific margins with itemized cost analysis
  • Time Range Filtering: Sophisticated date selectors for monthly, quarterly, yearly analysis
  • Profit Calculation Engine: Advanced algorithms for accurate margin calculations

🏪 Complete Business Management - OPERATIONAL

  • Order Tracking: Comprehensive order management with real-time status updates
  • Product Catalog: Full product database with variants, pricing, inventory tracking
  • Customer Management: Complete customer data, purchase history, communication logs
  • Expense Management: Categorized business expense tracking with tax preparation
  • Financial Reporting: Automated P&L statements and tax-ready financial reports

🔄 Automated Data Processing - FUNCTIONAL

  • CSV Import: Bulk import orders and transactions from Etsy statements
  • PDF Processing: Extract data from Etsy payment receipts and statements
  • Smart Product Matching: AI-powered matching of imported data to product catalog
  • Bulk Operations: Process hundreds of orders and transactions efficiently

🚧 Coming Soon - In Active Development

  • Interactive Charts: Chart.js components for visual profit trends and analytics
  • Product Performance Analysis: Detailed insights on most/least profitable products
  • Revenue Forecasting: Predictive analytics based on historical patterns
  • Seasonal Trend Analysis: Understanding business cycles and seasonal patterns

🛠 Tech Stack

Frontend

  • React 18 with TypeScript
  • Vite for fast development and building
  • Tailwind CSS for styling
  • Redux Toolkit for state management
  • React Router for navigation
  • Chart.js for data visualization
  • React Hook Form with Zod validation

Backend

  • Node.js with Express and TypeScript
  • MongoDB with Mongoose ODM
  • JWT authentication
  • bcryptjs for password hashing
  • Helmet for security
  • Express Rate Limit for API protection
  • Morgan for logging
  • Cors for cross-origin requests

📦 Project Structure

etsy-tracker/
├── client/                 # React frontend
│   ├── src/
│   │   ├── components/     # Reusable UI components
│   │   ├── pages/         # Page components
│   │   ├── store/         # Redux store and slices
│   │   ├── utils/         # Utility functions
│   │   └── ...
│   ├── public/
│   └── package.json
├── server/                 # Node.js backend
│   ├── src/
│   │   ├── controllers/   # Route handlers
│   │   ├── models/        # Database models
│   │   ├── routes/        # API routes
│   │   ├── middleware/    # Custom middleware
│   │   └── index.ts       # Server entry point
│   ├── .env.example
│   └── package.json
├── .github/
│   └── copilot-instructions.md
├── package.json           # Root package.json
└── README.md

🎯 DEPLOYMENT READY - Multiple Options Available

Perfect for Docker Desktop, Portainer, or similar tools

  1. Clone Repository:

    git clone https://github.com/dlawler489/etsy-finance-tracker.git
    cd etsy-finance-tracker
    
  2. Choose Your Deployment Method:

    • docker-compose.simple.yml - Best for beginners
    • docker-compose.deploy-local.yml - 🔧 Enhanced debugging
    • docker-compose.deploy.yml - 🚀 Pre-built images (fastest)
  3. Import & Deploy:

    • Docker Desktop: Compose tab → Import → Select file → Deploy
    • Portainer: Stacks → Add stack → Upload file → Deploy
    • Access: http://localhost:8081

Option 2: Command Line Deployment

# Quick deployment with pre-built images
docker-compose -f docker-compose.deploy.yml up

# Local build with enhanced debugging
docker-compose -f docker-compose.simple.yml up --build

Option 3: Development Environment

# For customization and development
npm install
cd client && npm install  
cd ../server && npm install
npm run dev  # Starts both frontend and backend

📋 Validation: Run ./validate-deployment.sh to check your environment

🏗️ DEVELOPMENT STATUS & ROADMAP

Phase 1 - COMPLETED (April 2026)

  • Core profit analysis dashboard with real-time metrics
  • Complete business management system (orders, products, customers)
  • Advanced data import and processing capabilities
  • Comprehensive deployment infrastructure with multiple options
  • Production-ready containerization with CI/CD pipeline

🚧 Phase 2 - IN PROGRESS (May 2026)

  • Interactive Chart.js components for visual analytics
  • Advanced product performance analysis with recommendations
  • Enhanced profit trending and forecasting capabilities

📋 Phase 3 - PLANNED (June 2026)

  • AI-powered business insights and recommendations
  • Advanced seasonal analysis and inventory optimization
  • Mobile app companion for on-the-go business monitoring cd etsy-finance-tracker
    
    
  1. Install dependencies

    cd client && npm install && cd ..
    cd server && npm install && cd ..
    
  2. Set up environment variables (optional)

    cd server
    cp .env.example .env
    # Edit .env if needed for development
    

Development

Option 1: Run both servers with Docker (recommended)

./build-deploy.sh local

Option 2: Run development servers separately

npm run dev

Option 2: Run servers separately

Start the backend server:

npm run server:dev
# Server runs on http://localhost:8080

Start the frontend development server:

npm run client:dev
# Client runs on http://localhost:3000 (or next available port)

Building for Production

# Build both client and server
npm run build

# Or build separately
npm run client:build
npm run server:build

Running Tests

npm test

🔧 Configuration

Environment Variables (Server)

Create a .env file in the server directory:

NODE_ENV=development
PORT=8080
CLIENT_URL=http://localhost:3000
MONGODB_URI=mongodb://localhost:27017/etsy-tracker
JWT_SECRET=your-super-secret-jwt-key-change-this-in-production
SESSION_SECRET=your-super-secret-session-key-change-this-in-production

Database Setup

The application uses MongoDB to store:

  • Products: Product details, variants, pricing, inventory
  • Orders: Order information, customer details, status tracking
  • Customers: Customer profiles, purchase history
  • Expenses: Business expense records
  • Users: Authentication and user management

📊 API Endpoints

Authentication

  • POST /api/auth/register - User registration
  • POST /api/auth/login - User login
  • POST /api/auth/logout - User logout
  • GET /api/auth/me - Get current user

Products

  • GET /api/products - Get all products
  • POST /api/products - Create new product
  • GET /api/products/:id - Get specific product
  • PUT /api/products/:id - Update product
  • DELETE /api/products/:id - Delete product

Orders

  • GET /api/orders - Get all orders
  • POST /api/orders - Create new order
  • GET /api/orders/:id - Get specific order
  • PUT /api/orders/:id - Update order status

Customers

  • GET /api/customers - Get all customers
  • POST /api/customers - Create new customer
  • GET /api/customers/:id - Get specific customer
  • PUT /api/customers/:id - Update customer

Expenses

  • GET /api/expenses - Get all expenses
  • POST /api/expenses - Create new expense
  • PUT /api/expenses/:id - Update expense
  • DELETE /api/expenses/:id - Delete expense

Analytics

  • GET /api/analytics/dashboard - Get dashboard metrics
  • GET /api/analytics/sales - Get sales analytics
  • GET /api/analytics/products - Get product performance
  • GET /api/analytics/customers - Get customer analytics

🎨 UI/UX

The application features a modern, responsive design built with Tailwind CSS:

  • Responsive Design: Works on desktop, tablet, and mobile devices
  • Dark/Light Mode: User preference support
  • Interactive Charts: Visual analytics and reporting
  • Form Validation: Real-time validation with helpful error messages
  • Toast Notifications: User feedback for actions

🔒 Security Features

  • JWT Authentication: Secure token-based authentication
  • Password Hashing: bcrypt for secure password storage
  • Rate Limiting: API endpoint protection
  • CORS Configuration: Cross-origin request handling
  • Input Validation: Server-side validation for all inputs
  • Security Headers: Helmet.js for security headers

🚀 Deployment

# Build the application
npm run build

# Install PM2 globally
npm install -g pm2

# Start the server with PM2
cd server
pm2 start dist/index.js --name "etsy-tracker-api"

# Serve the client (using a static file server)
pm2 serve client/dist 3000 --name "etsy-tracker-client"

Using Docker

# Build and run with Docker Compose
docker-compose up --build

🤝 Contributing

  1. Fork the repository
  2. Create a feature branch: git checkout -b feature/new-feature
  3. Commit your changes: git commit -am 'Add new feature'
  4. Push to the branch: git push origin feature/new-feature
  5. Submit a pull request

📝 License

This project is licensed under the MIT License - see the LICENSE file for details.

🆘 Troubleshooting

Common Issues

  1. Port already in use: If you get an EADDRINUSE error, kill the process using the port:

    lsof -ti:8080 | xargs kill -9
    
  2. MongoDB connection issues: Ensure MongoDB is running and the connection string is correct in your .env file.

  3. Build errors: Clear node_modules and reinstall:

    rm -rf node_modules client/node_modules server/node_modules
    npm run install:all
    

VS Code Integration

This project includes VS Code configuration for:

  • Tasks: Pre-configured build and dev tasks
  • Debugging: Launch configurations for both client and server
  • Extensions: Recommended extensions list
  • Settings: Project-specific settings

📞 Support

If you encounter any issues or have questions, please:

  1. Check the troubleshooting section above
  2. Search existing issues in the repository
  3. Create a new issue with detailed information about the problem

Built with ❤️ for Etsy sellers who want to grow their business through better data tracking and analysis.