Line drawing is known for its simplicity and minimalism. Artists usually struggle with abstract representations, and robots struggle much more. When creating an illustration, an artist must use abstraction and key visual features to capture the most crucial information. Abstraction entails determining the essential visual aspects of an item or scene, which requires semantic understanding and prior knowledge of high-level ideas.
An innovation would help examine a computer's capacity to simulate the abstract drawing process. Researchers have proposed an optimization-based photo-to-sketch technique. They're employing CLIP, a neural network trained on photos and text. The CLIP encoder provides a semantic comprehension of the exhibited issue, while the photo serves as a geometric foundation. They can achieve various levels of abstraction. A specific sketch dataset is not required for the proposed method to work. The research is highly relevant for Sketching Software Market as it presents a technique for drawing objects led by geometric and semantic simplifications.
Most sketch generation algorithms rely on explicit drawing datasets for training. On the other hand, Contrastive-Language-Image-Pretraining (CLIP) has the unique ability to infer semantic concepts from sketches and pictures alike. A sketch is described as a set of Bézier curves. The parameters of the curves are directly optimized against a CLIP-based perceptual loss. This is done through a differentiable rasterizer. The number of strokes controls the degree of abstraction. The resulting sketches include multiple layers of abstraction while maintaining the basic visual components, underlying structure, and subject's recognizability.
A differentiable rasterizer based on CLIP-based loss is used to optimize the stroke parameters. The finished sketch contains both semantic and aesthetic features to successfully express the essence of the information. The level is determined by the number of sketches used.
