viton-虚拟试衣

主要实现的功能是对于一张人体的图片进行换不同的衣服，人体可能处于不同的姿态，衣服的褶皱啥的都会有影响。

参考代码路径：https://github.com/pongsate1/fashion-parsing

http://vision.is.tohoku.ac.jp/~tangseng/clothing_parsing_project

一、项目背景：

虚拟试衣即实现用户及时不用脱去身上的衣服，也能实现变装查看效果的一种应用。虚拟试衣分为3D虚拟试衣和2D虚拟试衣两种；3D虚拟试衣的实现需要获得模特、服装等三维数据来实现，较为复杂，对于唯品会现在产品的层面来说，实现起来成本过高，不太现实；2D的试衣能通过用户直接调换想试装的衣服而达到试装的效果，从而达到提高转换率的目的。

三、算法规划：

versions	description	method	data	results	status	timestamp	milestone
V0.1 (baseline)	vanilla	https://arxiv.org/pdf/1711.08447.pdf	zalando_women_top	Input: Output:	DONE	05/15/2018	1 (a CVPR18 paper indicate it could be promising and applied in product)
in-house inference	do clothe transform on VIP model images				DONE	06/01/2018	1
sc-tps in python	python version of shape context and thin-plate-spline transformation in order to get rid of matlab code and able to run in server			a little worse than original matlab version	DONE	06/01/2018	1
enhance hand	make hand more real	add hand landmarks to input channel https://github.com/CMU-Perceptual-Computing-Lab/openpose		improved hand (result from stage 1)	DONE	06/15/2018	1
enhance face	make face more real	add face landmarks to input channel https://github.com/CMU-Perceptual-Computing-Lab/openpose		no improvement	DONE	06/15/2018	1
openpose service	make an openpose service in 139, so that people in other team could try out, etc	flask service		command line: curl -F "imagefile=@images/U63A0723-5.jpg"http://10.103.11.139:5001/api/v1/pose_estimation return: { "people": [ { "face_keypoints_2d": [ 96.6001, 39.4791, 0.777463, 96.6001, '''	DONE	06/15/2018	1
optimize training	experiment with different training configs to make the training / experimenting more efficeinet	grid service, code abstraction, build evaluation tools, etc			DOING expected results 6/26		1
crawl more data	crawl more data from zalando for training		1. crawled zalando_women_top 2. crawled zalando_women_men_top 3. combine with existing one 4. gradually add more categories		DOING expect 2 / 3 results 6/26		1
try end2end model	put the 2 stage model and the tps transformation together and make a unified end2end model.	Borrow the idea from Layered recursive GAN, https://openreview.net/pdf?id=HJ1kmv9xx instead of optimize with GAN, use perceptual loss with encoder-decoder	zalando_women_top		DOING		1
DensePose	densepose prior
improve collar	tricks to make collar more realistic	https://arxiv.org/pdf/1711.08447.pdf			NOT STARTED bonus		1
background	figure out a way to add / generate difference background	make background as conditional input? not sure yet			NOT STARTED		1
improve segmentation					NOT STARTED		1
add GAN objective	add GAN objective to improve performance	https://arxiv.org/pdf/1805.03356.pdf			NOT STARTED		1
channel wise feature alignment	align input channel (pose, segmentation) and output channel (rgb)	https://arxiv.org/pdf/1804.05827.pdf			NOT STARTED		1
try cycleGAN idea	make a cycle 1 product image -> transform -> model image 2 model image -> segmentation -> transform -> product image	https://arxiv.org/pdf/1703.10593.pdf			NOT STARTED		1
try attention mechanics	generate but conditioned on input image regions, both spatially and channel wise	https://arxiv.org/pdf/1711.10485.pdf			NOT STARTED bonus		1
do high-resolution generation	multi-scale / fine-grained generation	https://arxiv.org/pdf/1711.11585.pdf			NOT STARTED		1
full 2d transform service	if we get it so far, the results should be good enough in constrained domain, e.g., t-shirts => t-shirts; then we should make a real demo / service.	hoop up all the models and optimize speed etc			NOT STARTED		2 (at this point, it's useful service and potential product)
multi-view generation	input multi-view / output multiview	structure from motion or other multi-view methods			诗和远方		2
real-time service	real-time image service				诗和远方		2
interactive editing	edit face / clothes attributes, etc	https://arxiv.org/pdf/1711.09020.pdf			诗和远方		2
streaming		video			诗和远方		3 (cool product)