Recovering Geometric Information with Learned Texture Perturbations
Abstract
Regularization is used to avoid overfitting when training a neural network; unfortunately, this reduces the attainable level of detail hindering the ability to capture high-frequency information present in the training data. Even though various approaches may be used to re-introduce high-frequency detail, it typically does not match the training data and is often not time coherent. In the case of network inferred cloth, these sentiments manifest themselves via either a lack of detailed wrinkles or unnaturally appearing and/or time incoherent surrogate wrinkles. Thus, we propose a general strategy whereby high-frequency information is procedurally embedded into low-frequency data so that when the latter is smeared out by the network the former still retains its high-frequency detail. We illustrate this approach by learning texture coordinates which when smeared do not in turn smear out the high-frequency detail in the texture itself but merely smoothly distort it. Notably, we prescribe perturbed texture coordinates that are subsequently used to correct the over-smoothed appearance of inferred cloth, and correcting the appearance from multiple camera views naturally recovers lost geometric information.
Document Details
- Document Type
- Pub Defense Publication
- Publication Date
- Sep 22, 2021
- Source ID
- 10.1145/3480137
Entities
People
- Hui Zhou
- Jane Wu
- Ronald Fedkiw
- Yongxu Jin
- Zhenglin Geng
Organizations
- Epic Games
- JD.com
- Office of Naval Research
- Stanford University