CHARACTER ANALYSIS USING SPACE IN HANDWRITING IMAGE TO DETERMINE SPECIALIZATION IN BUSINESS
Sudiksha Chakraborty*, Jyoti Majumder
ICFAI Business School (IBS), Kolkata, India.
*Corresponding Author’s Email: sudiksha.chakraborty18@ibsindia.org
ABSTRACT
Specialization selection in business has been a subject of research study for several years. In our work, it begins with collecting the handwriting samples on plain white A4 size paper. Initially color image or gray scale image is taken as an input then thresholding is done to convert the image into binary image and noise removal technique is also applied. Skew- normalization techniques have been applied to find out space between lines, words and letters in handwriting images after segmentation of lines, words and characters have been performed. Finally, the mean of the space between all the closed loops formed by the characters has been found out and compared with the word spaces to determine the character. The characters are then matched with the requirements of each specialization. Accordingly, the candidates select their specialization and their performance in the exam has proved it to be a healthy match for them. This paper focuses on selection of specialization in business based on behavior from space analysis in handwritten document. The proposed method was tested on more than 500 text images of IAM database and sample handwriting images which are written by different writers on different backgrounds, detects the exact space in between lines, words and characters after skew normalization of a document. The experimental result shows that proposed algorithm achieves more than 63% accuracy for all type skew angles.
During the last few years specialization selection in business has been an intensive research topic. It can be done based on characteristics or behavior of an individual. The initial phase of business is dedicated towards completion of the core studies, designed to build a strong foundation of basic concepts across various areas of business. In the latter part, an individual can choose to specialize in an area. There may be numerous factors that influence the decision regarding the area of specialization. Depending upon the priority, the individual may choose the field as per the market demand, interest area, job profiles and the compensation packages offered by the companies. But prior to that the most important basis should be the parity of the skills required for a field with the characteristics or behavior of the person. The person would not be able to fulfil the expectations of the area or business if he/she lacks the skills. Adaptability being an alternative, could not deliver 100% in the case. This proposed work assumes that scanned handwriting document may only consist of Salt and Pepper noise and Background noise, caused by dirt on the document and can be removed by filters like median (Solihin & Leedham, 1997) and k-fill (Story et al., 1992) filter. Skew (Sarfraj & Rasheed, 2008; Bal & Saha, 2016; Panwar & Nain, 2012) in handwriting document can occur due to the human’s behavior as well as by the scanner during the scanning process. In the case of correct orientation of the pages, handwriting document still consist of smaller and
larger skew (Sarfraj & Rasheed, 2008; Bal & Saha, 2016; Panwar & Nain, 2012) due to writer variation. Thus, on its application, the results on the now rotated lines are found to be more accurate.
The next step segmentation (Nicolas, Paquet & Heutte, 2004; Marti & Bunke, 2001; Simistira, Papavassiliou & Stafylakis, 2011; Zimmermann & Bunke, 2002; Luthy, Varga & Bunke, 2007) which partitions the binary image into multiple segments depending on requirement. The present segmentation methods are K-mean segmentation (Nicolas, Paquet & Heutte, 2004; Marti & Bunke, 2001; Simistira, Papavassiliou & Stafylakis, 2011; Zimmermann & Bunke, 2002; Luthy, Varga & Bunke, 2007) and projection profiles (Nicolas, Paquet & Heutte, 2004) etc. The last step is classification (Otsu, 1979) method which is the heart of the handwriting recognition technique. This paper concentrates on selection of specialization through space analysis i.e. determination of behavior based on comparison of space between words and the space between the closed loops formed by the characters. Accordingly, the matching of those with the required skills for a specialization are done.
Technically skew is defined as the alignment of the text lines and words with respect to the horizontal direction. Skew can be either clockwise, called positive skew or anti- clockwise, called negative skew. But at the time of recognition process, handwriting document should be free from the skew for better recognition. Skew normalization is required for vertically character segmentation. There are four types of skew are available, that are Negative or Ascending skew, Positive or Descending skew, Normal or straight skew and Wavy skew, which are shown in figure 1.
Example of the positive skewed word and negative skewed words are shown in figure 2(a) and 2(b). The movement of the positive skew towards the clockwise direction and movement of the negative skew towards the anti-clockwise direction.
Let us consider a document having positive or negative skew. When the segmentation of lines, words and characters is performed, it is often found that due to too much slanting of lines, the top or bottom or both sides of the lines are getting cut-off. As a result, the space calculations in between lines, words and characters are getting deviated. In case of overlapping lines in the original document also, this segmentation is having errors. There are
numerous previous methods that have been proposed in past research i.e. Sarfraj & Rasheed (2008). Skew Estimation and Correction of Text using Bounding Box, Fifth IEEE conference on Computer Graphics, Imaging and Visualization, (Nicolas, Paquet & Heutte, 2004). Text Line Segmentation in Handwritten Document Using a Production System, Proceedings of the 9th Int’l Workshop on Frontiers in Handwriting Recognition (IWFHR-9), for skew detection and normalization of document image such as Global Hough Transform (Srihari & Govindaraju, 1989), Local Hough Transform (Srihari & Govindaraju, 1989), Projection Histogram (Otsu, 1979), etc. On applying any of these methods, skew normalization is possible.
The figure 3 shows the closed loop of letters in words of the handwriting image. The handwriting is written on a blank plain A4 sheet of paper. Wherever words like “and”, “gap”, “boy”, etc. are found, the proposed method detects the closed loop of “a”, “d”, “g”, etc. and calculates the radii of all such characters and stores in an array (Papavassiliou et al., 2010). Then the sum of the diameters of all such characters is calculated and their mean is found out. Now, this mean space between the loops of these characters is compared with the space between the words of that document. If double the loop space is found greater than the word space, then the person is narrow-minded and vice-versa and possesses some pre-defined qualities in previous papers (Kedar, Nair & Kulkarni, 2015; Champa & AnandaKumar, 2010; Das Gupta, & Chanda, 2012).
LITERATURE REVIEW
In 2007, Luthy, Varga & Bunke proposed segmentation of line using Hidden Markov Models (HMMs) and these models have been tested on the text line, where the aim was to be able to differentiate between inter- word gaps and intra-word gaps.
Marti & Bunke (2001) introduced a system for reorganization English handwriting text based on large vocabulary. The proposed method segments complete lines of text lines into single words. The algorithm tested on 541 text lines containing 3899 word and performed correct segmentation rate of 95.56%.
Story et al., in 1992 have identified an electronic based browsing system for recognition of the patterns from scanned image.
In 2015, Kedar, Nair & Kulkarni proposed a Personality Identification through Handwriting Analysis. In graphology, spacing implies the distance maintained between the lines, words and letters by the writer. Spacing can reveal various personality constructs. Spacing can be identified into two main groups as wide or narrow as shown in table 1 below:
In 2016, Bal & Saha proposed and discussed about skew normalization procedures. In 2012, Panwar & Nain proposed the amount of space left between one line and the next establishes how well she organizes her life. The method has been evaluated on various styles of handwritten text documents, and it is found that it detects the exact skew angle with more than 98% accuracy.
In 2010, Champa & AnandaKumar proposed a method that classified Human Behavior: it shows that the writer is independent, cautious and a reserved person, he may also be experiencing loneliness at the time he wrote the sample.
In 1997, Solihin & Leedham proposed Narrow spacing: the writer who uses close spacing needs constant contact with people that could make him sociable and may also have feelings of insecurity.
In 1979, Otsu proposed a Normal spacing: as usual whenever there is a balance in the element under examination then we can say that the person has got a balanced attitude.
In 2012, Das Gupta & Chanda proposed a slop or slant recognition theory to identify the effect of skew normalization.
Proposed Work
Generally, skew is measured depending on the baseline. Space calculation between lines, words and characters are done after skew normalization on sample document. According to the review (Kedar, Nair & Kulkarni, 2015; Champa & AnandaKumar, 2010; Das Gupta, & Chanda, 2012) in the previous section, there are several pre-defined behavioral characteristics analyzed based on spacing. Previously proposed methods (Kedar, Nair & Kulkarni, 2015; Champa & AnandaKumar, 2010; Das Gupta, & Chanda, 2012) only state the characteristics based on assumptions but do not match it with the calculations and thus it is tested here.
In the proposed work deal with identification of behavior traits with the help of space analysis comparing the space between the closed loops of letters with the space between the words and this proposed method is not concerned about the other characteristics of handwriting analysis.
At first, in our proposed work, the color handwriting documents are collected from surroundings, converted to gray scale handwriting document from the IAM database (Solihin, & Leedham, 1997; Story et al., 1992). Proposed approach assumes that the given handwriting documents are perfectly scanned so only the skews which are introduced by the writers are considered. Initially noise removal techniques (Solihin, & Leedham, 1997; Story et al., 1992) have been applied on the scanned handwriting document. Skew normalization (Sarfraj & Rasheed, 2008; Bal & Saha, 2016; Panwar & Nain, 2012) is applied on segmented lines to normalize them. Then this proposed method is applied on the segmented lines to find the space calculations between lines, words and characters on the modified skewed document to get accuracy.
In the proposed method for calculating the space between lines, words and characters, it is required to find out first the space between the words by segmentation (Nicolas, Paquet & Heutte, 2004; Simistira, Papavassiliou & Stafylakis, 2011; Zimmermann & Bunke, 2002; Luthy, Varga & Bunke, 2007) and projection profile (Nicolas, Paquet & Heutte, 2004). Now, the mean of the diameters of all closed looped characters is calculated. This mean space between the loops of these characters is compared with the space between the words of that document (Champa & AnandaKumar, 2010). The details approach for space calculations according to original and skewed documents are explained in the experimental section.
After all, the behavior traits received from comparison of the space calculated from their samples and the mean of the diameters are matched with the pre-defined ones required for selecting a specialization.
Skew Normalization on the document:
This is applied on the document to make it free from positive and negative skews and convert it into a straight line.
Steps:
1. if (x1>x2)
2. % negative skew 3. t =x1;
4. x1=y1;
5. y1=t;
6. t =x2;
7. x2=y2;
8. y2=t;
9. y2=y1+y1-y2;
10. slope=(y2-y1)/(x2-x1);
angle=atand(slope);
rot=imrotate (new, -angle);
imshow(rot);
[rr,cc]=size(rot);
rend=rstart+rr-1;
rotated (rstart: rend,1:cc, :)=rot;
rstart=rend+1;
else
%positive skew
20. t=x1;
21. x1=y1;
22. y1=t;
23. t=x2;
24. x2=y2;
25. y2=t;
26. y2=y1+y1-y2;
27. slope=(y2-y1)/(x2-x1);
angle=atand(slope);
rot=imrotate(new,angle);
[rr,cc]=size(rot);
rend=rstart+rr-1;
rotated (rstart: rend,1:cc,:)=rot;
rstart=rend+1;
end
end
imshow(rotated);
Line spacing calculation on skewed document:
Steps:
hor<- horizontal_projection(rotated); // hor is an array which stores sum of pixels of a row
for (each element in hor)
if value>10000 // checking presence of written pixel start of new line
for (each element in hor) 5. if(value>10000)
else
line ends;
store starting and ending points in “lines” matrix
End-if
Exit inner loop;
Continue outer;
Lines (3)-lines (2) gives the space between Line no. 1 and Line no. 2, lines(5)-lines(4) gives the space between Line no. 2 and Line no. 3
using this we calculate total space and the “total space/total linecount” operation gives the line space mean result.
In the above Algorithm (Algorithm 2) we have calculated the space between the lines on the skewed document.
Word spacing calculation on skewed document:
Steps:
linestart=1; pos=size_of(lines) // no. of lines*2
while(linestart<=pos) do
new=img(lines(linestart): lines(linestart+1)) // assigning image of a line in new matrix
ver<-vertical_projection(new) //finding vertical projection in order to separate words
for (each element in ver)
if(value>450)//checking presence of written pixel
hash(i)=hash(i-1) +1;//hash matrix defines consecutive written pixels depicting a word
else
hash(i)=0;
endif
endfor
word_extract(new); //extraction of words
wordstart=1; posw=size_of(words) //words contains starting and ending position of all words in the line
while(wordstart<=posw) do
new_word=img(words(wordstart): words(wordstart+ 1)) // assigning image of a line in new matrix
totalwordspace=totalwordspace+words(wordstart)- words(start-1) //calculation of word space
char_extract(new_word);
In the above Algorithm (Algorithm 3) we have calculated the space between the words of a line in the original document. During word extraction, we are storing the starting column pixel and the ending column pixel in an array named “Words”. For e.g., Words (1) and Words (2) depict the starting and ending column pixel respectively of Word no. 1 and likewise Words (3) and Words (4) depict word no. 2. Now we can calculate the space between words by calculating the difference between Words (3) and Words (1).
Character spacing calculation on skewed document:
Steps:
ver<-vertical_projection(new_word) //finding vertical projection in order to separate words
For (each element in ver)
if(value>450)//checking presence of written pixel
hashw(i)=hashw(i-1) +1; //hash matrix defines consecutive written pixels depicting a character
else
hashw(i)=0;
endif
endfor
/*same steps as in word_extract and here we store characters in “chars” array. */
/* calculation of space*/
start=3;
while(start<=posc) //posc signifies no. of chars in the word
totalcharspace=totalcharspace+chars(start)- chars(start- 1);
endwhile
In the above Algorithm (Algorithm 4), we have calculated the space between the characters of a word in the original document. During character extraction, we are storing the starting column pixel and the ending column pixel in an array named “Chars”. For e.g., Chars (1) and Chars (2) depict the starting and ending column pixel respectively of Character no. 1 and likewise Chars (3) and Chars (4) depict word no. 2. Now we can calculate the space between characters by calculating the difference between Chars (3) and Chars (1).
Calculation of radii of the closed loops of the letters in a skewed document:
Steps:
rgb=imread(img);
gray_image=convertToGrayScale(img);
[centers,radii]= imfindcircles(gray_image, [130],‟Object Polarity‟,‟dark‟,‟Sensitivity‟,0.9);
[centersBright,radiiBright]= imfindcircles(rgb, [130],‟Object Polarity‟,‟bright‟,‟Sensitivity‟,0.9);
In the above Algorithm (Algorithm 5) we use the function imfindcircles () to detect circular objects within the radius range of 1 to 30 units. By the “bright” keyword we mean all the bright circular objects which we eventually take into the calculations of mean (refer to table 2).
Function Name | Function Description |
imfindcircles | Find out the closed loops of word image |
hash | To implement hash matrix |
horizontal_projection | Horizontal Projection |
lines | Assigning image of a line in a new matrix |
vertical_projection | Vertical Projection to separate words |
word_extract | To extract words |
char_extract | To extract characters |
rotate | Rotate the image with given degree |
The proposed work implemented in MATLAB on IAM database (Nicolas, Paquet & Heutte, 2004) over 500 text images containing 3500 words and some sample handwriting image which are written by different writer on different background. Previous proposed algorithm can measure any amount of skew angle and take the appropriate normalization approach and now proposed method can calculate the space between the lines, words and characters on the original document and on the skewed document from these segmented lines.
The text line shown in figure 4 contained the negative skew and after applying skew normalization method where line is totally balanced and normalized with respect to baseline shown in figure 5. After lines segmentation and normalization, the words are segmented from the text lines and two ample of segmented words are shown in figure 6(a) and 6(b). Training set of all the experiments has been carried out on total number of 500 text image and sample handwriting image by different writer on different background. The experimental result of skew normalization process shows in figure 5 and figure 6(a) and (b).
The above figures 6(a) and 6(b) contains extracted word and character from skew normalized document respectively after applying the skew normalization method. The space between the lines, words and characters are thus calculated from these images.
The significant efficiency to deal with the overlapping sample, negative or positive skewed samples, etc. is shown in table 3.
The original text sample along with patterns of closed loop letters recognized from it is provided in the figures 7(a) and 7(b) below.
From the above table 4, it is cleared that proposed method is much efficient for behavior analysis and can deal with any types of skew angle up to 360º. If double the diameter mean is less than word space after skew normalization, the person is broad-minded and vice-versa (Story et al., 1992). The success rate for this process of the proposed method is 63% better than the other methods. More results with details are shown in table 2. The failure cases of the proposed method are mainly caused by misclassified to lower case letters.
From table 5, each category of people and their detailed character traits are described.
In table 6, the skills and personality required for each domain in business is shown. On comparing those features with the already available traits, we specify the type of specialization each one should acquire or take up in business.
In table 7, the performance of the people selecting their choice of specialization is shown. The performances have been good when the people select the correct specialization according to their character analysis and vice-versa.
DISCUSSION
Initially, the handwritten manuscripts in A4 sheet papers are collected from different writers in the environment. Then the scanned images are used as input to the model and the noise removal algorithms are applied to make the image noise-free. Then K-mean segmentation came into play to convert the color image into gray scale and form clusters from it for better identification of the lines by extracting the image of lines from the background.
Since, many a times, the lines of a document are not aligned to the same baseline, skew normalization becomes necessary to get more accurate results about one’s characteristics. It basically rotates the lines to remove the skew angle and brings them to the same baseline. This is followed by segmentation methods and projection profile algorithms. They are used to extract the lines, words and characters (Nicolas, Paquet & Heutte, 2004; Simistira, Papavassiliou & Stafylakis, 2011; Zimmermann & Bunke, 2002; Luthy, Varga & Bunke, 2007). The space between the words are thus calculated in pixels. The mean of the spacing between the words is required for comparison.
The whole handwritten image is used for identifying the closed loop letters by pattern recognition. The diameter mean of all the closed loops are calculated and matched against the mean word space to understand the nature of the person. This was further granularized into character elements of a person. This was important to understand which specialization in management will suit the person best by matching the characteristics with the requirement of the job to be expected out of it in the future so that the one can do justice to it. Their performances throughout the course has been analyzed and matched to validate that the prediction exactly fits their performance.
CONCLUSION
The method has been proposed to select the desired specialization in business based on the writer’s handwriting in current scenario. Using the proposed method 96% lines and words were normalized perfectly with very small error rate and the character analysis based on space calculation is found with 63% accuracy. This method deals with exact classification of the inner traits and match them with the specialization best suited for them. It is very efficient to compare to existing technique and can work with skew up to 360º.
The future work can include more handwriting features with the proposed method removing the overlapping loops detected and some constant factor and obtain the more robust system.
ACKNOWLEDGMENT
The authors are very much grateful to the IBS, Kolkata for giving the opportunity to work on ‘Character Analysis Using Space in Handwriting Image to Determine Specialization in Business’. All the authors sincerely express their gratitude to Dr. Ajay Pathak, Director of IBS, Kolkata for giving constant encouragement in doing research in the field of innovation of digital trends in business methods.
REFERENCES
Bal, A. & Saha, R. (2016). An Improved Method for Text Segmentation and Skew Normalization of Handwriting Image. Progress in Intelligent Computing Techniques: Theory, Practice, and Applications, pp 181-196. Retrieved From: https://link.springer.com/chapter/10.1007/978-981-10-3373-5_18
Champa, H.N. & AnandaKumar, K.R. (2010). Automated Human Behavior Prediction through Handwriting Analysis, 2010 First International Conference on Integrated Intelligent Computing. 5th-7th August, IEEE Computer Society, pp 160-165. Retrieved From: https://ieeexplore.ieee.org/document/5571472
Das Gupta, J. & Chanda, B. (2012). Novel Methods for Slope and Slant Correction of Off- line Handwritten Text Word, 2012 Third International Conference on Emerging Applications of Information Technology (EAIT), 30th Nov.- 1st Dec. IEEE, pp 295-298. Retrieved From: https://ieeexplore.ieee.org/document/6407927
Kedar, S., Nair V. & Kulkarni S. (2015). Personality Identification Through Handwriting Analysis: A Review. International Journal of Advanced Research in Computer Science and Software Engineering, 5(1), pp 548-556.
Luthy, F., Varga, T. & Bunke H. (2007). Using Hidden Markov Models as a Tool for Handwritten Text Line Segmentation, Ninth International Conference on Document Analysis and Recognition (ICDAR 2007), 23rd-26th September. Retrieved From: https://ieeexplore.ieee.org/document/4378666
Marti, U.V. & Bunke, H. (2001). Text Line Segmentation and Word Recognition in a System for General Writer Independent Handwriting Recognition. Proceedings of Sixth
International Conference on Document Analysis and Recognition. IEEE, 13th September. Retrieved From: https://ieeexplore.ieee.org/document/953775
Nicolas, S., Paquet, T. & Heutte, L. (2004). Text Line Segmentation in Handwritten Document Using a Production System. Proceedings of the 9th Int’l Workshop on Frontiers in Handwriting Recognition (IWFHR-9), IEEE, pages 6. Retrieved From: http://www.iapr- tc11.org/archive/iwfhr2004_proc/docs/041_x_nicolas-TextLi.pdf
Otsu, N. (1979). A Threshold Selection Method from Gray Level Histogram. IEEE Transactions on Systems, Man, and Cybernetics, 9(1), pp 62-66.
Panwar, S. & Nain N. (2012). A Novel Approach of Skew Normalization for Handwritten Text Lines and Words, 8th International Conference on Signal Image Technology and Internet Based Systems, SITIS, pp 296-299. Retrieved From: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.885.9881&rep=rep1&type=pdf
Papavassiliou, V., Stafylakis T., Katsouros V. & Carayannis G. (2010). Handwritten Document Image Segmentation into Text Lines and Words. Pattern Recognition, 43(1), pp 369-377.
Sarfraj, M. & Rasheed, Z. (2008). Skew Estimation and Correction of Text using Bounding Box. Fifth IEEE conference on Computer Graphics, Imaging and Visualization, pp. 259–264. Retrieved From: https://www.semanticscholar.org/paper/Skew-Estimation-and-Correction- of-Text-Using-Box-Sarfraz-Rasheed/3b61fa3c17f082da0ec7b21ffd1936b8f76b675c
Simistira, F., Papavassiliou, V. & Stafylakis, T. (2011). Enhancing Handwritten Word Segmentation by Employing Local Spatial Features. 2011 International Conference on Document Analysis and Recognition, IEEE, pp 1314-1318. Retrieved From: http://www.iapr- tc11.org/archive/icdar2011/fileup/PDF/4520b314.pdf
Solihin, Y. & Leedham, C.G. (1997). Noise and Background Removal from Handwriting Images, Institute of Electrical and Electronics Engineers (IEEE), 8-10th December. pp 366-370. Retrieved From: https://ieeexplore.ieee.org/document/645299
Srihari, S.N. & Govindaraju, V. (1989). Analysis of Textual Image using the Hough Transform. Machine Vision and Applications, 2(3), pp 141–153.
Story, G., O’Gorman, L., Fox, D., Schaper, L.L. & Jagadish, H.V. (1992). The Right Pages Image-based Electronic Library for Alerting and Browsing. IEEE Computer, 25(9), pp 17-26.
Zimmermann, M. & Bunke, H. (2002). Automatic Segmentation of the IAM Off-line Database for Handwritten English Text, IEEE, pages 6. Retrieved From: file:///C:/Users/LUC/Downloads/Automatic_Segmentation_o_the_IAM_OffLine_Database.p df