BrunoGaoandClaude
4da23af857
- Add RSS news crawler with 5 international medical tech news sources - Configure GitHub Actions for automated bi-daily news updates - Implement smart data loading with fallback mechanism - Clean up mock data (reduced from 55+ items to 1 fallback item) - Add comprehensive documentation and troubleshooting guide - Create verification and testing scripts Key Features: - Real-time news from MedTech Dive, Medical Device Network, etc. - Automated crawling via GitHub Actions (8:00 & 20:00 Beijing time) - Graceful degradation when crawler data unavailable - Date parsing fixes and User-Agent configuration - 20 real news articles successfully crawled Documentation: - NEWS_CRAWLER_USAGE.md - User guide - NEWS_CRAWLER_IMPROVEMENTS.md - Implementation details - REAL_NEWS_CRAWLER_DESIGN.md - Architecture design - TROUBLESHOOTING.md - Debug guide - test-news-data.html - Browser testing tool 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
40 lines
965 B
Bash
Executable File
40 lines
965 B
Bash
Executable File
#!/bin/bash
|
|
|
|
# 真实新闻爬虫快速测试脚本
|
|
|
|
echo "🚀 开始测试真实新闻爬虫..."
|
|
echo ""
|
|
|
|
# 1. 安装依赖
|
|
echo "📦 安装爬虫依赖..."
|
|
cd scripts
|
|
npm install --silent
|
|
cd ..
|
|
|
|
# 2. 运行爬虫
|
|
echo "🕷️ 运行爬虫抓取新闻..."
|
|
cd scripts
|
|
npm run crawl
|
|
|
|
# 3. 检查结果
|
|
echo ""
|
|
echo "📊 检查抓取结果..."
|
|
if [ -f "../public/data/news.json" ]; then
|
|
NEWS_COUNT=$(cat ../public/data/news.json | grep -o '"id"' | wc -l)
|
|
echo "✅ 成功抓取 $NEWS_COUNT 条新闻"
|
|
|
|
echo ""
|
|
echo "📰 最新5条新闻标题："
|
|
cat ../public/data/news.json | jq -r '.[:5] | .[] | " - \(.title)"'
|
|
|
|
echo ""
|
|
echo "📈 新闻来源统计："
|
|
cat ../public/data/news.json | jq -r 'group_by(.source) | .[] | " \(.[0].source): \(length)条"'
|
|
|
|
echo ""
|
|
echo "✅ 测试完成！访问 http://192.168.1.83:5175/website/industry-news 查看效果"
|
|
else
|
|
echo "❌ 未找到新闻数据文件"
|
|
exit 1
|
|
fi
|