This work is about a system to help novice users create animations quickly. The system is called K-Sketch and is a 2D animation sketching system.
In this system users first sketch objects and next create animations for those drawn objects such as translation, scaling and rotation.
This is similar to Flash’s tween capabilities and can be incorporated there
This work is a sketch-based game for collecting data on how people make and describe sketches. The system described collects drawings and attach the meta description the users provided.
To score points in this game, users must reconstruct a drawing based on a text description or vice-versa. In another phase, users are asked to judge the work of other players.
This paper is quite similar to the previous sketch game paper. But the idea is still fine.
This paper presents basically an image search where the input image is a sketch. The sketch is actually the edges of the image to be searched for. The feature of the images to which the sketch is compared is edge histogram.
It took 3 seconds to search a 1.5 million image bank which is a good performance I think.
The image comparison and image search solutions are becoming more widely used and those which rely on the edges can use sketch as their query since a sketch can be considered the edges of an image
The authors implemented a multi player sketching game in order to gather sketch data. One is called Picturephone and the other is Stellasketch.
In Picturephone, one player describes a sketch and the other draws it. In Stellasketch, players label a sketch drawn by a user individually. Finally, players rate various drawings to identify similarities
In Stellasketch a player draws something based on a clue and other players identify the sketch privately.
Both games seem fun to play especially Picturephone which is a sketch version of the telephone game.
This paper is about a Tablet PC application called InkSeine which supports active note taking by coupling a pen-and-ink interface with an in situ search facility that flows directly from a user’s ink notes.
Their user study showed users interest in having search results available in the context of surrounding notes and appreciate the ability to break searches down into multiple episodes so as to maintain attention on the primary note taking taks
This paper is a good human centered design project. I also think abandoning mouse and keyboard in favor of pen based interface must be accelerated.
Yet another paper on generating 3D models from 2D sketches.However this one is even more fun. It not only let the user to generate 3D meshes, but also let them alter its texture by which the user has a better idea of the final product.
This system was used to create new plush toys and was also tested on small kids
It is amazing that children connected to this program and it shows success
The work in this paper a user is asked to draw a 2D sketch which is essentially a 3D projection. Next a 3D model is created from the projection.
Their user study shows that the system is robust and is able to perform the intended task quickly
Well it is more a streovision paper than a sketch recognition.
In this paper, Mathpad2 is introduced which lets users to do math. Gestures are used to help segmenting the formulas and to help identification. It is also able to generate graphs and plots. It is also possible to change stroke color to help organization
There are definitely room to improve the system such as less reliance on gestures, nevertheless the system looks cool.
In this paper, the authors propose a method of grouping related sketches which does not use timing data.
They essentially search in the space of all possible groupings of strokes, however they incorporate some pruning mechanisms to avoid intractability. One of such pruning is to only group strokes in close proximity. Another measure is restricting the size of such group.
Finally they create an image out of each group and use an adaboost image processing approach to see if it is anything meaningful.
This paper is actually taking advantage of the successful application of Viola et al. Adaboost method to face detection. Since it is a very fast algorithm (not in training), they have the freedom of running an expensive search to try different combinations of the sketches to do the grouping.
This system is supposed to distinguish between text and graphics. What they do is to first extract a set of features out of strokes, such as it direction, length width ratio of it, total curvature, etc. Subsequently, it takes uses features of the gaps between strokes and finally by incorporating an HMM, they put all these in context of a sequence.
Their experiments demonstrated the usefulness of temporal context however the advantage of incorporating gap information is not evident in their results
As they mention in the paper, they have ignored the length of the gaps, so it would be better to regard stroke within which there is a large gap, independent. However using a temporal model, in this case HMM for the text and graphics seemed reasonable and worked well too
This paper talk about a graph-based symbol recognizer. The authors used a graph called Attribute Relational Graph. The authors try to use such graph and find isomorphism.
The graph represents the topology of the primitive shapes in the graph. In their user study they collected several types of symbols and used their four matching algorithms on the data. The results were from around 68% to 98%
I think they could have used better search approaches to this problem
A multi stroke hand drawn symbol recognizer is proposed here. Their technique shows a way to learn symbols definitions using prototype examples allowing users to train new symbols.
Their method has two steps. First, polar coordinates are used to determine angular alignment and eliminate unlikely definitions. Next, the surviving definitions are tested using the normal screen coordinates with four template classifiers. The results of individual classifiers are combined to produce the recognizer’s final decision.
They test their system using two user study. First, on numeric digits where they performed only slightly worse than dedicated recognizers. The second test was in the engineering symbols domain such as resistors, transistors, integral symbol, etc. Their accuracy for being in the top 2 of the result was above 96% in all the tests they conducted in this domain.
This paper used some interesting distance measures and ideas which I think I could use earlier in my projects if I had known them.
In this paper entropy rates are proposed as a discriminator between text and shape. Since text strokes usually show more curvature change, the authors have utilized this to distinguish them from other shapes.
Their accuracy of classification reached around 92% which is fine for a relatively simple method.
It is a more decent an different approach than the decision tree which was discussed in previous papers.
In this work, the authors propose the use of constellation models in sketch recognition which is similar to the technique used in computer vision.
In their work, they label all the constituent parts of the sketch. For instance, it it is a sketch of a face, the objective is to find which part is the mouth, which is the nose, so on. To do so, they extract the parameters of the strokes distribution and the stroke pairs’ distribution from the training set. Next, to classify a sketch, all possible labeling are tried to find the one with maximum likelihood.
The idea of using constellation model in sketch recognition is interesting however one limitation of this method is the requirement for each part of the sketch being labeled with e.g. eye, mouth, etc. I think they have made a big assumption about the availability of such tags.
Also LADDER can be thought of as a complement or a rival to this method as it can support more complex relationships, however in LADDER, relationships are defined whereas here, the relationships are learned.
In this paper a sketch recognition language is proposed in which template objects are described by their component primitives and geometric constraints between them; For instance, a stick figure consists of a circle connected to a line, which is itself connected to four other lines.
The shapes consist of parts such as lines, constraints such as being parallel or intersections, aliases, editing and display properties.
Once the structural description have been entered using LADDER language, they can be automatically transformed into shape recognizers.
The paper brings examples of applying the framework to UML diagrams. The framework, also encompass a library of predefined shapes (line, curve, ellipse) and constrains ( collinear, tangent, acute ).
LADDER is a tool which is valuable to many domains as long as the time it takes to precisely define all the shapes is justified. However as part of its future work, it can pose automatic description generation for complex shapes.
This paper introduces HUNCH system. It has a similar corner finding mechanism to that of Sezgin’s, that is , it find the minima in the speed curve. It also consider connects line endpoints using a connectivity threshold. Finally it experiments with some high level recognition such as 3D object inference or floor plan recognition. Their system employed CURVIT and STRAIT modules as curve recognizer and line endpoint latching module respectively.
They conclude that in order to have a successful sketch recognition system, some context information is necessary.
This is a relatively old paper, however they propose some characteristics of a sketch system which is still ideal today, that is to identify sketches without context information. This paper is mostly experiments and possible solutions. Experiments and question proposed are still relevant today.
In this paper a system is introduced to recognize a set of primitive shapes. The primitive shapes recognized by this paper are: straight lines, polylines, arcs, circles, helixes, spirals, etc.
It is a low level process which is usually utilized in the initial steps of the recognition to provide input to higher levels.
The recognizer basically gives the strokes to a set of different low level recognizers, namely eight. Each classifier then returns whether the stroke is belongs to that class or not along with the beatified stroke in case it is.
Moreover, PaleoSketch incorporates two new features: normalized distance between direction extremes (NDDE) and direction change ratio (DCR), both of which are a measure to detect spikes in the direction graph and are discriminators between Curve and Polyline.
PaleoSketch has a very high classification rate with overall recognition of about 98.6%, it paves way for higher level recognitions. However, I think the only important primitives to be recognized are circle, line and polyline. Shapes such as spiral or helix are quite domain dependent and the choice to add them to the list of primitive shapes seems arbitrary to me.
This paper focuses on the initial steps of sketch recognition. It proposes a method to find corners in a sketch. Sezgin combines both curvature and speed measures to generate a set of potential corners.
The second feature in their method is a curve handling. It model curves as Bezier curves by approximating the control points using LS method.
The system also performs stroke beautification so the small noises of the pen while drawing the shapes are eliminated and as a result the drawn sketches will look better
Finally, the system does some basic object recognition on circles, rectangles, etc.
This paper is one of the earliest in corner detection which takes advantage of both speed and curvature. About the segmentation, use of Bezier curves for segments is a good choice for its flexibility. Finally the results are promising
This method was compared against Dynamic Time Warping method and Rubine’s one were the rotation and scale invariance mechanisms were applied to the input of all these methods even without incorporating the Golden Section Search. The test set contains a set of 16 stroked drawn in different speed and by different people.
The DTW and the method proposed here produced similar and better result than that of Rubine’s especially when few samples were available for each template. Regarding the performance, DTW was significantly slower than Rubine’s and the method proposed here.
The limitations mentioned on the paper are the inability to distinguish between shapes whose difference is based on aspect ratio, location, scale, position or speed of drawing. However except speed, by not performing some of initial steps the issue is resolved.
Another weak point of the $1 recognizer is that it compares the candidate gesture to all templates at runtime which has performance implications. Nevertheless, in case of a large template set, templates tend to collide in the feature space and this is where the feature-based methods fail as well.
