From 058fb1a85e460f58623fa02f5723bcdb6493dc7d Mon Sep 17 00:00:00 2001 From: Winifred <100462076+Winifred43@users.noreply.github.com> Date: Mon, 22 Aug 2022 16:23:38 +0800 Subject: [PATCH 1/2] Update README.md --- README.md | 29 +++++++++++++++++------------ 1 file changed, 17 insertions(+), 12 deletions(-) diff --git a/README.md b/README.md index 9e3f2bae..b81a7f21 100644 --- a/README.md +++ b/README.md @@ -33,27 +33,32 @@ tips: | :----- | :---- | :---- | | linux | base | Custom stand-alone operating environment, free to implement all custom stand-alone functions | | datax | import export | Import and export of heterogeneous data sources | -| media-download | data processing | Distributed download of media files | -| video-audio | data processing | Distributed extraction of audio from video | -| video-img | data processing | Distributed extraction of pictures from video | +| hadoop | data processing | hdfs,hbase,sqoop,spark client | | sparkjob | data processing | spark serverless | +| volcanojob | data processing | volcano multi-machine distributed framework | | ray | data processing | python ray multi-machine distributed framework | -| volcano | data processing | volcano multi-machine distributed framework | -| xgb | machine learning | xgb model training and inference | | ray-sklearn | machine learning | sklearn based on ray framework supports multi-machine distributed parallel computing | -| pytorchjob-train | model train | Multi-machine distributed training of pytorch | -| horovod-train | model train | Multi-machine distributed training of horovod | -| tfjob | model train | Multi-machine distributed training of tensorflow | +| xgb | machine learning | xgb model training and inference | +| tfjob | deep learning | Multi-machine distributed training of tensorflow | +| pytorchjob | deep learning | Multi-machine distributed training of pytorch | +| horovod | deep learning | Multi-machine distributed training of horovod | +| paddle | deep learning | Multi-machine distributed training of paddle | +| mxnet | deep learning | Multi-machine distributed training of mxnet | +| kaldi | deep learning | Multi-machine distributed training of kaldi | | tfjob-train | model train | distributed training of tensorflow: plain and runner | | tfjob-runner | model train | distributed training of tensorflow: runner method | | tfjob-plain | model train | distributed training of tensorflow: plain method | -| kaldi-train | model train | Multi-machine distributed training of kaldi | | tf-model-evaluation | model evaluate | distributed model evaluation of tensorflow2.3 | | tf-offline-predict | model inference | distributed offline model inference of tensorflow2.3 | -| model-offline-predict | model inference | distributed offline model inference of framework | -| deploy-service | model deploy | deploy inference service | +| model-register | model service | register model to platform | +| model-offline-predict | model service | distributed offline model inference of framework | +| deploy-service | model service | deploy inference service | +| media-download | multimedia data processing | Distributed download of media files | +| video-audio | multimedia data processing | Distributed extraction of audio from video | +| video-img | multimedia data processing | Distributed extraction of pictures from video | +| object-detection-on-darknet | machine vision | object-detection with darknet yolov3 | +| ner |natural language | Named Entity Recognition | - # Deploy [wiki](https://github.com/tencentmusic/cube-studio/wiki/%E5%B9%B3%E5%8F%B0%E5%8D%95%E6%9C%BA%E9%83%A8%E7%BD%B2) From bdc65bbe9680de7cea4792008172018900c2be27 Mon Sep 17 00:00:00 2001 From: Winifred <100462076+Winifred43@users.noreply.github.com> Date: Mon, 22 Aug 2022 16:28:58 +0800 Subject: [PATCH 2/2] Update README_CN.md --- README_CN.md | 41 ++++++++++++++++++++++++----------------- 1 file changed, 24 insertions(+), 17 deletions(-) diff --git a/README_CN.md b/README_CN.md index 54c1a1a5..0c0d0030 100644 --- a/README_CN.md +++ b/README_CN.md @@ -32,26 +32,33 @@ https://github.com/tencentmusic/cube-studio/wiki | :----- | :---- | :---- | | 自定义镜像 | 基础命令 | 完全自定义单机运行环境，可自由实现所有自定义单机功能 | | datax | 导入导出 | 异构数据源导入导出 | -| media-download | 数据处理 | 分布式媒体文件下载 | -| video-audio | 数据处理 | 分布式视频提取音频 | -| video-img | 数据处理 | 分布式视频提取图片 | +| hadoop | 数据处理 | hadoop大数据组件，hdfs,hbase,sqoop,spark | | sparkjob | 数据处理 | spark serverless 分布式数据计算 | | ray | 数据处理 | python ray框架 多机分布式功能，适用于超多文件在多机上的并发处理 | -| xgb | 机器学习 | xgb模型训练 | -| ray-sklearn | 机器学习 | 基于ray框架的sklearn支持算法多机分布式并行计算 | | volcano | 数据处理 | volcano框架的多机分布式，可自由控制代码，利用环境变量实现多机worker的工作与协同 | -| pytorchjob-train | 训练 | pytorch的多机多卡分布式训练 | -| horovod-train | 训练 | horovod的多机多卡分布式训练 | -| tfjob | 训练 | tf分布式训练，k8s云原生方式 | -| tfjob-train | 训练 | tf分布式训练，内部支持plain和runner两种方式 | -| tfjob-runner | 训练 | tf分布式-runner方式 | -| tfjob-plain | 训练 | tf分布式-plain方式 | -| kaldi-train | 训练 | kaldi音频分布式训练 | -| tf-model-evaluation | 模型评估 | tensorflow2.3分布式模型评估 | -| tf-offline-predict | 离线推理 | tf模型离线推理 | -| model-offline-predict | 离线推理 | 所有框架的分布式模型离线推理 | -| deploy-service | 服务部署 | 部署云原生推理服务 | - +| ray-sklearn | 机器学习 | 基于ray框架的sklearn支持算法多机分布式并行计算 | +| xgb | 机器学习 | xgb模型训练 | +| tfjob | 深度学习 | tf分布式训练，k8s云原生方式 | +| pytorchjob | 深度学习 | pytorch的多机多卡分布式训练 | +| horovod-train | 深度学习 | horovod的多机多卡分布式训练 | +| horovod | 深度学习 | horovod 的多机多卡分布式训练 | +| paddle | 深度学习 | paddle的多机多卡分布式训练 | +| mxnet | 深度学习 | mxnet的多机多卡分布式训练 | +| kaldi | 深度学习 | kaldi的多机多卡分布式训练 | +| tfjob-train | tf分布式 | tf分布式训练，内部支持plain和runner两种方式 | +| tfjob-runner | tf分布式 | tf分布式-runner方式 | +| tfjob-plain | tf分布式 | tf分布式-plain方式 | +| kaldi-train | tf分布式 | kaldi音频分布式训练 | +| tf-model-evaluation | tf分布式 | tensorflow2.3分布式模型评估 | +| tf-offline-predict | tf分布式 | tf模型离线推理 | +| model-register | 模型服务化 | 注册模型 | +| model-offline-predict | 模型服务化 | 所有框架的分布式模型离线推理 | +| deploy-service | 模型服务化 | 部署云原生推理服务 | +| media-download | 多媒体处理 | 分布式媒体文件下载 | +| video-audio | 多媒体处理 | 分布式视频提取音频 | +| video-img | 多媒体处理 | 分布式视频提取图片 | +| object-detection | 机器视觉 | 基于darknet yolov3 的目标识别| +| ner | 自然语言| 命名实体识别 | # 平台部署