文章目录准备工作训练自己的数据集开始训练训练日志可视化准备工作参考链接https://pjreddie.com/darknet/yolo/下载darknet修改Makefile下载yolov3.weights下载darknet53.conv.74后面训练数据集会用到先下载好了留着备用测试#1. 下载darknetgit clone https://github.com/pjreddie/darknet#2. 修改Makefilecd darknet make#3. 下载yolov3.weightswget https://pjreddie.com/media/files/yolov3.weights#4. 下载darknet53.conv.74wget https://pjreddie.com/media/files/darknet53.conv.74#5. 测试图片./darknet detect cfg/yolov3.cfg yolov3.weights data/dog.jpg./darknet detector test cfg/coco.data cfg/yolov3.cfg yolov3.weights data/dog.jpg#5. 测试视频./darknet detector demo cfg/coco.data cfg/yolov3.cfg yolov3.weightsvideofile#5. 加载摄像头./darknet detector demo cfg/coco.data cfg/yolov3.cfg yolov3.weights总结可以将上述下载的文件备份一下以后每次自己训练数据集的时候就不用再次下载了直接改用。训练自己的数据集参考链接https://www.cnblogs.com/answerThe/p/11481564.html在darknet文件夹下新建一个myData文件夹。将标注好的图片和xml文件放到对应目录下。运行test.py生成train.txt/val.txt/test.txt/trainval.txt文件。myData包含如下文件夹myData......JPEGImages#存放图像......Annotations#存放图像对应的xml文件......ImageSets/Main#存放训练/存放train.txt/val.txt/test.txt/trainval.txt文件......test.py#生成train.txt/val.txt/test.txt/trainval.txt文件test.py代码如下importosimportrandom trainval_percent0.1train_percent0.9xmlfilepathAnnotationstxtsavepathImageSets\Maintotal_xmlos.listdir(xmlfilepath)numlen(total_xml)listrange(num)tvint(num*trainval_percent)trint(tv*train_percent)trainvalrandom.sample(list,tv)trainrandom.sample(trainval,tr)ftrainvalopen(ImageSets/Main/trainval.txt,w)ftestopen(ImageSets/Main/test.txt,w)ftrainopen(ImageSets/Main/train.txt,w)fvalopen(ImageSets/Main/val.txt,w)foriinlist:nametotal_xml[i][:-4]\nifiintrainval:ftrainval.write(name)ifiintrain:ftest.write(name)else:fval.write(name)else:ftrain.write(name)ftrainval.close()ftrain.close()fval.close()ftest.close()如果按照上述文件结构则test.py文件不需要修改直接运行即可生成txt文件。将darknet文件夹下的scripts/voc_label.py拷贝出来修改成my_labels.py放在darknet文件夹中。【注意修改类别和路径】运行该脚本my_lables.py会在./myData目录下生成一个labels文件夹一个txt文件(myData_train.txt)(内容是: 类别的编码和目标的相对位置)。my_labels.py代码如下importxml.etree.ElementTreeasETimportpickleimportosfromosimportlistdir,getcwdfromos.pathimportjoin sets[(myData,train),(myData,val),(myData,train),(myData,val),(myData,test)]classes[person,foot,face]# 改成自己的类别defconvert(size,box):dw1./(size[0])dh1./(size[1])x(box[0]box[1])/2.0-1y(box[2]box[3])/2.0-1wbox[1]-box[0]hbox[3]-box[2]xx*dw ww*dw yy*dh hh*dhreturn(x,y,w,h)defconvert_annotation(year,image_id):in_fileopen(myData/Annotations/%s.xml%(image_id))out_fileopen(myData/labels/%s.txt%(image_id),w)treeET.parse(in_file)roottree.getroot()sizeroot.find(size)wint(size.find(width).text)hint(size.find(height).text)forobjinroot.iter(object):difficultobj.find(difficult).text clsobj.find(name).textifclsnotinclassesorint(difficult)1:continuecls_idclasses.index(cls)xmlboxobj.find(bndbox)b(float(xmlbox.find(xmin).text),float(xmlbox.find(xmax).text),float(xmlbox.find(ymin).text),float(xmlbox.find(ymax).text))bbconvert((w,h),b)out_file.write(str(cls_id) .join([str(a)forainbb])\n)wdgetcwd()foryear,image_setinsets:ifnotos.path.exists(myData/labels/):# 改成自己建立的myDataos.makedirs(myData/labels/)image_idsopen(myData/ImageSets/Main/%s.txt%(image_set)).read().strip().split()list_fileopen(myData/%s_%s.txt%(year,image_set),w)forimage_idinimage_ids:list_file.write(%s/myData/JPEGImages/%s.jpg\n%(wd,image_id))convert_annotation(year,image_id)list_file.close()注意这里面如果采用上述文件结构只需要将classes改成自己的类别即可其他内容不需要修改。在myData文件夹下新建myData.names文件。在myData文件夹下新建weights文件用于保存生成的权重文件。修改darknet/cfg下的voc.data和yolov3-voc.cfg文件。复制这两个文件并分别重命名为my_data.data和my_yolov3.cfg1修改my_data.dataclasses 4 #改为自己的分类个数 ##下面都改为自己的路径 train /home/zhan/darknet/myData/myData_train.txt valid /home/zhan/darknet/myData/myData_test.txt names /home/zhan/darknet/myData/myData.names backup /home/zhan/darknet/myData/weights2修改my_yolov3.cfgCtrlF搜出3个含有yolo的地方。每个地方都必须要改2处filters 、classesfilters3*5lenclasses可修改random 1原来是1显存小改为0。是否要多尺度输出。一般地max_batches修改成合适的数值。开始训练参考链接https://blog.csdn.net/csdn_zhishui/article/details/85397380训练首先将cfg/my_yolov3.cfg文件中改成Training模式。训练命令./darknet detector train cfg/my_data.data cfg/my_yolov3.cfg darknet53.conv.74# 指定gpu训练默认使用gpu0查看GPU情况nvidia-smi./darknet detector train cfg/my_data.data cfg/my_yolov3.cfg darknet53.conv.74-gups0,1,2,3# 训练过程中保存训练日志xxx.log./darknet detector train cfg/my_data.data cfg/my_yolov3.cfg darknet53.conv.74|tee train_yolov3.log# 断点继续训练./darknet detector train cfg/my_data.data cfg/my_yolov3.cfg myData/weights/my_yolov3.backup|tee new_train_yolov3.log训练日志可视化vis_yolov3_log.py代码如下# -*- coding: utf-8 -*-importpandasaspdimportmatplotlib.pyplotaspltimportos# 可能需要修改的地方#g_log_pathtrain_yolov3.log# 此处修改为自己的训练日志文件名# #defextract_log(log_file,new_log_file,key_word): :param log_file:日志文件 :param new_log_file:挑选出可用信息的日志文件 :param key_word:根据关键词提取日志信息 :return: withopen(log_file,r)asf:withopen(new_log_file,w)astrain_log:forlineinf:# 去除多gpu的同步logifSyncinginline:continue# 去除nan logifnaninline:continueifkey_wordinline:train_log.write(line)f.close()train_log.close()defdrawAvgLoss(loss_log_path): :param loss_log_path: 提取到的loss日志信息文件 :return: 画loss曲线图 line_cnt0forcount,lineinenumerate(open(loss_log_path,rU)):line_cnt1resultpd.read_csv(loss_log_path,skiprows[iter_numforiter_numinrange(line_cnt)if((iter_num500))],error_bad_linesFalse,names[loss,avg,rate,seconds,images])result[avg]result[avg].str.split( ).str.get(1)result[avg]pd.to_numeric(result[avg])figplt.figure(1,figsize(6,4))axfig.add_subplot(1,1,1)ax.plot(result[avg].values,labelAvg Loss,color#ff7043)ax.legend(locbest)ax.set_title(Avg Loss Curve)ax.set_xlabel(Batches)ax.set_ylabel(Avg Loss)defdrawIOU(iou_log_path): :param iou_log_path: 提取到的iou日志信息文件 :return: 画iou曲线图 line_cnt0forcount,lineinenumerate(open(iou_log_path,rU)):line_cnt1resultpd.read_csv(iou_log_path,skiprows[xforxinrange(line_cnt)if(x%39!0|(x5000))],error_bad_linesFalse,names[Region Avg IOU,Class,Obj,No Obj,Avg Recall,count])result[Region Avg IOU]result[Region Avg IOU].str.split(: ).str.get(1)result[Region Avg IOU]pd.to_numeric(result[Region Avg IOU])result_iouresult[Region Avg IOU].values# 平滑iou曲线foriinrange(len(result_iou)-1):iouresult_iou[i]iou_nextresult_iou[i1]ifabs(iou-iou_next)0.2:result_iou[i](iouiou_next)/2figplt.figure(2,figsize(6,4))axfig.add_subplot(1,1,1)ax.plot(result_iou,labelRegion Avg IOU,color#ff7043)ax.legend(locbest)ax.set_title(Avg IOU Curve)ax.set_xlabel(Batches)ax.set_ylabel(Avg IOU)if__name____main__:loss_log_pathtrain_log_loss.txtiou_log_pathtrain_log_iou.txtifos.path.exists(g_log_path)isFalse:exit(-1)ifos.path.exists(loss_log_path)isFalse:extract_log(g_log_path,loss_log_path,images)ifos.path.exists(iou_log_path)isFalse:extract_log(g_log_path,iou_log_path,IOU)drawAvgLoss(loss_log_path)drawIOU(iou_log_path)plt.show()可视化这部分除了需要将训练日志文件名修改成自己的还要特别注意skiprows[iter_num for iter_num in range(line_cnt) if ((iter_num 500))]和skiprows[x for x in range(line_cnt) if (x % 39 ! 0 | (x 5000))]这两部分需要根据自己的训练次数来设定的。分别表示迭代次数小于500次的跳过画图不用从501开始画图每隔39个数或者前5000个数跳过说白了就是前5000个数值舍弃从第5001个数开始每隔39个数取一个数值参与画图。附录两张非常令人糟心的图因为不知怎么地它就失败了