Content
# News Aggregator API (MCP Server)
A modular, scalable news aggregation backend built with MCP server conventions and TypeScript. This API provides a unified interface to TheNewsAPI for retrieving current and historical news articles with advanced filtering capabilities. Designed specifically for consumption by AI agents, this API prioritizes structured data and consistent patterns.
## Features
- **Unified API**: Consistent interface for accessing news data
- **TypeScript**: Fully typed for reliable development
- **MCP Architecture**: Scalable, modular design with separation of concerns
- **Multiple Endpoints**: Access various types of news content
- **Environment Variables**: Secure configuration management
- **Error Handling**: Consistent error formats with appropriate status codes
- **Filtering**: Support for various filtering parameters
- **Caching**: Intelligent caching system with management endpoints
- **Interactive Documentation**: Swagger-based API documentation
- **AI-Friendly Responses**: Structured data optimized for machine consumption
## Endpoints
### News Endpoints
| Endpoint | Description | Example |
|----------|-------------|--------|
| `/api/news/top` | Get top news headlines | `/api/news/top?categories=business` |
| `/api/news/all` | Get all news with advanced search | `/api/news/all?search=technology` |
| `/api/news/similar/:uuid` | Get similar articles to a specific one | `/api/news/similar/cc11e3ab-ced0-4a42-9146-e426505e2e67` |
| `/api/news/uuid/:uuid` | Get a specific article by UUID | `/api/news/uuid/cc11e3ab-ced0-4a42-9146-e426505e2e67` |
| `/api/news/sources` | Get available news sources | `/api/news/sources?language=en` |
### Cache Management Endpoints
| Endpoint | Description | Example |
|----------|-------------|--------|
| `/api/cache/stats` | Get cache statistics | `/api/cache/stats` |
| `/api/cache/clear` | Clear all cache | `DELETE /api/cache/clear` |
| `/api/cache/clear/:type` | Clear cache by type | `DELETE /api/cache/clear/top` |
### Utility Endpoints
| Endpoint | Description |
|----------|-------------|
| `/health` | Health check endpoint |
| `/docs` | Interactive API documentation |
| `/docs.json` | OpenAPI specification |
| `/examples` | Usage examples |
## Testing
This project implements a comprehensive testing strategy with multiple layers of tests to ensure reliability and correctness.
### Test Types
1. **Unit Tests**
- Isolated tests for individual components
- Mocked dependencies for true unit isolation
- Focused on business logic validation
2. **Controller Tests**
- Verify controller behavior with mocked services
- Test proper request handling and response formatting
- Confirm error handling patterns are consistent
3. **Integration Tests**
- Test the complete request-response cycle
- Verify API endpoints with supertest
- Mock external API calls for reliability
### Testing Best Practices
This project follows these testing best practices:
- **Isolation** - Tests don't depend on each other or external services
- **Repeatable** - Tests produce the same results on any environment
- **Fast** - Mocking external dependencies keeps tests efficient
- **Comprehensive** - Core functionality has high test coverage
- **Maintainable** - Tests follow consistent patterns and structure
### Running Tests
```bash
# Run all tests
npm test
# Run tests with coverage
npm run test:coverage
# Run a specific test file
npx jest path/to/test.ts
```
## Installation
```bash
# Clone the repository
git clone <repository-url>
# Install dependencies
npm install
# Set up environment variables
cp .env.example .env
# Edit .env file with your own API key and database URL
# Generate Prisma client
npx prisma generate
# Build the project
npm run build
# Start the server
npm start
```
## Configuration
Create a `.env` file with the following variables:
```
# News Aggregator API Environment Variables
# API Configuration
NEWS_API_TOKEN=your_api_token_here
# Server Configuration
PORT=3000
NODE_ENV=development
# Database Configuration
DATABASE_URL="file:./dev.db"
```
### Database Configuration
The application uses Prisma ORM with SQLite by default. The `DATABASE_URL` environment variable should point to your database file. For production, you can configure it to use PostgreSQL or MySQL.
## Development
```bash
# Run in development mode with hot reload
npm run dev
# Run linting
npm run lint
# Run tests
npm test
```
## Testing Infrastructure
The project includes a comprehensive test suite built with Jest and Supertest, covering API endpoints, database integration, and server connectivity checks.
### Running Tests
```bash
# Run all tests
npm test
# Run only API endpoint tests
npm run test:api
# Run only database integration tests
npm run test:db
# Run tests with coverage report
npm run test:coverage
# Run tests in watch mode (for development)
npm run test:watch
# Check server connectivity
npm run check-server
```
### Test Structure
- **API Endpoint Tests**: Located in `src/__tests__/api/`, these tests verify that all API endpoints return the expected responses, handle errors correctly, and apply appropriate filtering.
- **Database Integration Tests**: Located in `src/__tests__/database.test.ts`, these tests verify database connectivity, query execution, and transaction support.
- **Server Connectivity Script**: Located in `scripts/check-server.ts`, this script tests connectivity to a running server instance, verifying that critical endpoints are functioning correctly.
### Test Environment
Tests use a separate environment configuration specified by setting `NODE_ENV=test`. This ensures tests don't interfere with development or production environments. The setup includes:
- Automatic database connection setup and teardown
- Cache clearing between tests to ensure isolation
- Server startup and shutdown for endpoint testing
## Future Enhancements Todo List
### Priority 1: Foundation Improvements
- [x] **Caching System**
- Implement in-memory caching
- Cache frequently accessed data like top news and sources
- Add cache invalidation strategies
- Configure TTL (Time-To-Live) for different content types
- Add cache management endpoints
- [x] **Data Persistence Layer**
- Add database integration (SQLite with Prisma)
- Create schema for storing enhanced article metadata
- Implement repository pattern for data access
- Add migration system for schema changes
- [x] **API Documentation**
- Generate OpenAPI/Swagger specification
- Add interactive API documentation
- Create usage examples for each endpoint
## API Documentation
The API now includes comprehensive documentation using OpenAPI/Swagger:
- **Interactive API Documentation**: Available at `/docs` when the server is running
- **OpenAPI Specification**: JSON format available at `/docs.json`
- **Usage Examples**: Available at `/examples` with code samples in multiple languages
### How to Access the Documentation
1. Start the server with `npm run dev` or `npm start`
2. Open your browser to `http://localhost:3000/docs` for the interactive Swagger UI
3. Browse through the available endpoints, request parameters, and response formats
4. Reference implementation examples at `http://localhost:3000/examples`
### API Documentation Features
- Complete endpoint descriptions with parameters and response types
- Interactive "Try it out" capability to test endpoints directly
- Request/response schema definitions
- Code snippets for API consumption
- Examples of integrating with the MCP server architecture
### Priority 2: Security & Performance
- [ ] **Authentication & User Management**
- Implement JWT-based authentication
- Create user registration and login endpoints
- Add role-based access control
- Implement secure password handling
- [ ] **Rate Limiting**
- Add request throttling middleware
- Implement tiered access levels
- Create fair usage policies
- Add rate limit headers in responses
- [ ] **Monitoring & Analytics**
- Set up request logging and monitoring
- Add performance metrics collection
- Create usage dashboards
- Implement error tracking and alerting
### Priority 3: Enhanced Functionality
- [ ] **Content Processing**
- Add text summarization capabilities
- Implement entity extraction
- Add sentiment analysis
- Create topic categorization beyond source categories
- Implement duplicate detection and grouping
- [ ] **Personalization**
- Add user preference tracking
- Create personalized endpoints based on reading history
- Implement topic following functionality
- Add recommendation engine
- [ ] **Webhooks & Real-time Updates**
- Implement webhook registration
- Create event-based notification system
- Add SSE (Server-Sent Events) for real-time updates
- Implement topic subscription functionality
## Contributing
Contributions to the News Aggregator API project are welcome! This project is designed to be a friendly, well-documented codebase for those interested in MCP server architecture, especially with AI consumption in mind.
### How to Contribute
1. Fork the repository
2. Create a new feature branch (`git checkout -b feature/amazing-feature`)
3. Make your changes
4. Run tests to ensure everything works (`npm test`)
5. Commit your changes (`git commit -m 'Add some amazing feature'`)
6. Push to the branch (`git push origin feature/amazing-feature`)
7. Open a Pull Request
### Code Style
Please follow the existing code style and patterns in the project. This project uses ESLint for code quality and formatting.
### Security Considerations
When contributing, please ensure:
- No API keys or credentials are hardcoded
- Environment variables are used for configuration
- No sensitive data is logged
- Input validation is properly implemented
### Testing
All new features should include appropriate tests. Please follow the existing testing patterns in the project.
- [ ] **Search History & Bookmarks**
- Add saved searches functionality
- Implement article bookmarking
- Create reading history tracking
- Add user collections feature
## Contributing
1. Fork the repository
2. Create your feature branch: `git checkout -b feature/my-new-feature`
3. Commit your changes: `git commit -am 'Add some feature'`
4. Push to the branch: `git push origin feature/my-new-feature`
5. Submit a pull request
## License
ISC
