Journal of Signal and Information Processing, 2013, 4, 4351 doi:10.4236/jsip.2013.43B008 Published Online August 2013 (http://www.scirp.org/journal/jsip) 43 Color Texture Image Inpainting Using the Non Local CTV Model Jinming Duan1, Zhenkuan Pan1, Wangquan Liu2, XueCheng Tai3 1College of Information Engineering, Qingdao University, China; 2Department of Computing, Curtin University, Australia; 3Dep artment of Mathematics, University of Bergen, Norway. Email: duanmujinming@126.com, zkpan@qdu.edu.cn, w.liu@curtin.edu.au, tai@mi.uib.no Received April, 2013. ABSTRACT The classical TV (Total Variation) model has been applied to gray texture image denoising and inpainting previously based on the non local operators, but such model can not be directly used to color texture image inpainting due to cou pling of different image layers in color images. In order to solve the inpainting problem for color texture images effec tively, we propose a non local CTV (Color To tal Variation) model. Technically, the proposed model is an extension of local TV model for gray images but we take account of the coupling of different layers in color images and make use of concepts of the nonlocal operato rs. As the coupling of differen t layers for color images in the proposed model will in crease computational complexity, we also design a fast Split Bregman algorithm. Finally, some numerical experiments are conducted to validate the performance of the proposed model and its algorithm. Keywords: Color Texture Images; Image Inpainting; NLCTV Model; TV Model; The Split Bregman Algorithm 1. Introduction Image inpainting, sometimes referred as image comple tion or disocclusion, has become one of the fundamental problems in image processing and computer vision due to its broad applications in image editing, compression, object removal from a scene, text or scratch removal and old photo restoration etc.. Its aim is to restore the tar nished/missing parts of a broken image using th e existin g data, which is a typical illposed problem in applied mathematics. There are several possible ways to solve this problem and here we focus on solving this problem with Partial Differential Equations (PDEs) or variational methods due to its outstanding performance [14,6,811]. In this branch of methodology, the exiting approaches can be classified into two categories: the geometryori ented methods [211] and the textureoriented methods [1222,27]. For inpainting those gray nontexture images with small scale broken areas, researchers in [2] proposed a third order PDE model to propagate the surrounding in formation to the inpainted regions based on a heat diffu sion equation. Authors in [3] coped with the similar problems by solving NavierStokes equations and inves tigators in [4] proposed total variation (TV) inpainting model inspired by [5]. In [6] the researchers studied the same problem for binary image via using the CahnHil liard model. For inpainting those gray nontexture im ages with large scale broken areas, a third order PDE method is proposed in [8] based on curvaturedriven dif fusions (CDD) mechanism, and in fact this work is in spired by the concept of elastic and principle of con tinuation in computer vision [7]. Also researches in [911] solved the same problem using the variational framework including elastic terms. The abovementioned approaches are all the geome tryoriented methods and they only used local informa tion, but failed to solve the problems of broken texture inpainting and object removal. The inpainting problem for texture images must resort to the textureoriented methods. Up to now, there are three categories of meth ods to solve the inpainting problem for texture images. The first type of approaches is the texture synthesis pro posed in [12] based on patches, which has been acted as the fundamental tool for texture image inpainting, and also numerous examplarbased variational models [1316] have been proposed using the patch similarities. The second type of approach is based on image decomposi tion in the original domain or transformed frequency domain. For example, the investigators in [1719] inves tigated texture image inpainting with the geometry and texture information separately. Typically, in [18], the images to be inpainted are first transformed into the framelet domain so that it is represented by a set of framelet coefficients; then one performs thresholding on Copyright © 2013 SciRes. JSIP
Color Texture Image Inpainting Using the Non Local CTV Model 44 framelet coefficients to propagate the information from outside of broken region into it. Finally, one transforms it back to image domain. This frameletbased image in painting can remove the random noise in an image and enhance the edges and high frequency features of the image. This is one of the state of the art algorithms cur rently and we will use it for comparison in our experi ments. The third type of approaches for texture image inpainting comes from the idea of non local means (NLM) for image denoising with texture preserving [20]. In [21] the authors not only defined the non local gradient and divergence systematically, but also proposed the NLTV (Non Local Total Variation) models as extensions to their local counterparts for image denoising and inpaint ing. The elegance for the non local operators in [21] leads to very similar manipulations between non local models and local models in image processing. For exam ple, the authors in [22] used TV with non local graphs for gray image inpainting and superresolution, and re searchers in [23] applied the NLTV for gray image de blurring. Moreover, authors in [24] used NLTV for compressive sensing and [25], [26] proposed some image enhancement models involving an NLTV regularization term. Though the above mentioned algorithms are all based on TV model for image restoration, there is an other way for image restoration based on MumfordShah model [28], which is usually used for image segmenta tion. The authors in [27] combined the MTV regularizer term and non local operators in [21] to implement this Gammaconvergence approximated model for color tex ture image restoration. This motivates us to consider problem in this paper. As we know, such segmenta tionbased image restoration model suffers from two problems. First, it contains two variables: one is a piece wise smooth function used for the approximation of the original image, and the other is a piecewise function that represents the image edge and equals 0 on the edge sets and 1 on the smooth region, which makes the numerical implementation complicated, which logically results a low computation efficiency. Second, there are three pen alty parameters setting up in this model, so the choice of those parameters will become more difficult. In conclu sion, there are too many parameters to solve and tune in such model. The NLTV model on ly includes one variable and one penalty parameter in its energy functiona l and this model always brings much easier computational process and also provides excellent results in preserving texture of gray images. However, to our best of knowledge, this popular model has not been used for color image in painting. In this paper, we will focus on the NLTV model and revise this model for inpainting color texture image by combining the non local operators and CTV (Color Total Variation) model proposed in [29]. The proposed model can make use of the good performance of the non local operators in texture image processing as demonstrated for gray images and CTV model in edge preserving for color images. We observed from [31] that the TV model can preserve edges for denoised images, but would fail to preserve color image edges when used to defuse different layers separately. However, the Mul tichannel Total Variation (MTV) [30] and CTV [29] can have excellent performance in color image edge preserv ing. Numerous experiments reported in [31] demon strated that the CTV model outperforms the MTV model, so we adopt the CTV regularizer as the foundation for the proposed model in this paper. We also observed from [31] that CTV has higher complexity than MTV in im plementation. In order to overcome the disadvantage of low efficiency associated with CTV in the proposed model, we will also design a fast Split Bregman algo rithm for the proposed model. Technically, the standard Split Bregman algorithm for TV model in [32] is redeveloped through introducing an auxiliary variable and a Bregman iterative parameter with an aim to transform the original model into two simple sub problems. These two sub problems can be solved via alternating optimization technique; further the previous EulerLagrange equation with curvature associ ated with the CTV regularizer term is replaced with a simple one only associated with the Laplacian [31]. Though a generalized soft thresholding formula is de rived in an analytical form for TV model in [32], and this simplifies the computation complexity sig nificantly there. However, we noted that for the Split Bregman algorithm of the proposed CTV model, the exact generalized soft thresholding formula can not be derived as elegantly as in [32], and here we design an approximate one to sim plify the calculation s . In summary, the contributions of this paper can be summarized as follows. First, we propose a model which can solve the inpainting problem for color texture images by using the CTV model in [29] and the non local opera tors in [21]. This model combines the advantages of the CTV and the nonlocal operators nicely and can produce a high performance. Second, In order to improve the im plementation efficiency of the proposed model, we de velop a fast Split Bregman algorithm based on a simple discrete finite difference scheme. Also we found that the direct application of NLTV model to color images for inpainting does not work properly. Finally, we validate the proposed model and algorithm via extensive experi ments. 2. NLTV Model for Image Inpainting and Its Split Bregman Algorithm In this section, we first introduce some important con Copyright © 2013 SciRes. JSIP
Color Texture Image Inpainting Using the Non Local CTV Model 45 cepts, definitions and technical algorithms used in the remaining parts of this paper. First we present the con cept of nonlocal operators. 2.1. The Non Local Operators and the Split Bregman Algorithm Due to the significance of the non local operators played in this paper, we first present the relevant definitions provided in [21]. Let n be the domain of a gray image andx , :ux R is a real function defined on to represent the pixel values of an image. The non local gradient for two points x and y in the image is defined as ,, NLuxyuy uxwxy (1.1) where, ,:wxyR is a nonnegative, symmetric weight between points , for any pair y ,xy and it measures the similarities of these two points. It should be noted that eq (1.1) is not a vector field in the standard sense, it is only a mapping: . Now we denote any NL mapping as ,:y Rpp x. For a pair of NL mappings, their dot product is defined as follows. 121 2 ,,pp xpxypxydy (1.2) And their inner product is defined as 121 212 ,,1,,pppppx ypxydxdy (1.3) The magnitude of a NL mapping will be given by 2 ,pxpppxy dy (1.4) With the above inne r product, the non lo cal diverg ence : NL px will be defined as the adjoint of the non local gradient, which is given by ,,, NL pxpxy pyxwxydy (1.5) Finally, the Laplacian of a point x in an image can be defined now by 1 2 , NLNL NL ux ux uyux wxydy (1.6) Based on the above mentioned d efinitions, we can give the NL norm of the NL gradient for a function u as follows. 2, NLuxuy ux wxydy (1.7) All above preliminaries are for a function of an image. Next we explain the NLTV models for gray images and their Split Bregman algorithm. For a broken scalar texture image : xR, let denote the domain to be inpainted, the proposed NLTV model for a gray image inpainting reported in [21] is given as follow. D 2 1 2 NL D u inE uux dxufdx (2) where, 0 1/ D D x D is the mask function to represent the broken region. This problem aims to find u in the already known broken region D such that (2) is minimized. The computation of local TV model utilizing the non local operators becomes very expensive, which will be demonstrated in experiment section. In order to improve its computational efficiency, authors in [21] designed a dual method for NLTV models. Such dual metod is not suitable for our proposed NLCTV model in this paper due to a fact that it involves complicated coupling feature in CTV regularizer term. Here, we alternatively extend the Split Bregman algorithm reported in [32] to our proposed NLCTV model which is much easier than the dual method. The author in [33] proposed the Split Bregman algorithm for NLTV denosing model. Here, we first present their algorithm for the NLTV inpainting model as reported in [32]. To present the Split Bregman algorithm for (2), an NL auxiliary variable ,:vvxyR is introduced and the objective function is transformed into the following: 2 , 1 ,2D uv inEu vvxdxufdx (3) s.t. NL vu (4) The constraint NL vu is enforced using the efficient Bregman iteration by introducing a Bregman parameter ,:ybbx R as reported in [32]. Then we can transform (3) into the following iterative optimization formulation. 2 2 1 1 ,2 2 D k NL Euvv xdxufdx vubxdx (5) with constraints 10 , kk kk NL bb uvbv 0 0 . By using the same technique as reported in [32], first fixing v for u, and then fixing for v, we can obtain the solution of EulerLagrange equation of u and the gener alized soft thresholding formula of via such alternat ing minimization process as stated in (6) and (7). A fast approximate solution of (6) is provided by a Gauss Seidel iterative scheme, and it is very convenient to find an analytic solution of (7) without any iteratio n. However, when it comes to NLCTV model, such exact soft thresholding formula as (7) can not be directly derived that we could not extend the efficient Split Bregman u v Copyright © 2013 SciRes. JSIP
Color Texture Image Inpainting Using the Non Local CTV Model 46 algorithm to our proposed model directly. Therefore, in the next section, we will focus on this tough problem. 10 kk DNLNL ufvub (6) 11 111 1 1,0 kk kkk NL NL kk NL ub vMaxubub 1 (7) 2.2. The NLCTV Model and Its Split Bregman Algorithm The previous model (2) is for gray images and if we use (2) directly to different layers separately for color image denoising or inpainting, and we found that this will lead to smear edges as demonstrated in Figure 6, though it can be solved via coupled regularizers such as MTV regularizer [30] or CTV regularizer [29]. The MTV and CTV have excellent performance in color image edge preserving as reported in [31]. However, numerous ex periments reported in [31] demonstrated that the CTV model outperforms the MTV model, so we adopt the CTV regularizer as the foundation for the proposed model. Therefore, in this paper, we extend the CTV to NLCTV model for color texture image inpainting. For color image denoising, the authors in [29] have proposed the following CTV model 22 11 2 nn ii uii i inE uuxdxufdx (8) where 12 ,,..., n ff f ,..., n u u is the original image, 12 is the restored image. Based on these results, we propose the following NLCTV model by combining (2) and (8) for color texture image inpainting. ,uu 22 11 1 2 nn NL iDii uii inE uuxdxufdx (9) It should be noted that (2) is an NLTV model and can only use to deal with the inpainting problem for gray images. By taking account of the coupling of different layers of color images and introducing the coupled NLCTV regularizer term 2 1 n NLi i uxdx (10) which is inspired by (8), we proposed the model (9). One can see that the differences between models (8) and (9) are as follow: First, model (8) is a denosing model and (9) is a inpainting model, and in (8) is a penalty parame ter that ensures that the denoised image is as close as possible to the original image; but in (9) is a mask function that labels the broken region of image. Second, model (8) replaces the local CTV regularizer term 2 1 n i i uxdx (11) with NLCTV regularizer term (10) by using the non local gradient operator L. Such choice is done since the NLCTV regularizer term will lead to an extremely complicated Eulerequation similar to (9), which is very difficult for discrete numerical calculation. u In order to solve (9) efficiently, we need to design a new Split Bregman algorithm similar to that reported in [32]. For such purpose and by using the same manner as reported in last section, we introduce an auxiliary variable 12 ,,..., n vvvv and a Bregman iterative parameter 12 ,,..., n bbb b , and then transform (9) into the following iterative optimizati on formulation. 22 11 2 1 1 1 ,2 2 nn iDi ii nk iNLii i i uvvxdxufdx vubxdx 0 . (12) With constraints 10 ,0 kk kk iiNLiiii bb uvbv v By using the alternating minimization strategy, we can ob tain the EulerLagrange equations for u and sepa rately as follows. 10 kk DiiNL iNLi i ufv ub (13) 11 2 1 0 i kk i iNLi ini i i vxdxv vubvx vxdx (14) In order to show how to implement (13) in detail, we give the discrete version of (13) as follows. 10 kk Dl llNLNLNLNLl ufvub (15) According to (1.5) and (1.6), we have ,, , kkk NLlh hllh lh vvv w (161) 111 ,, , kkk NLlh hllh lh bbb w (16.2) , 2 NLNLhlh l lh uuu w (16.3) where the nonnegative weight ,wxy is chosen as 2 2 ,exp Gux uy wxyr (17) In which G is the Gaussian kennel function, is the thresholding parameter for similarities between two patch windows, and its discrete version is given by r 2 ,2 exp lh Guluh wr (18) Copyright © 2013 SciRes. JSIP
Color Texture Image Inpainting Using the Non Local CTV Model 47 In order to present the algorithm clearer, we use the following diagram (Figure 1) to demonstrate the idea. When given a point l in the image, we can have a square search window and a patch window, in which l is center point of them. h represents any pixel point in the search window. When h is fixed in the search window, a square patch window is created subsequently and the weight ,lh between l point and h point can be computed as (18), in which their respectiv e patch windows are needed. w Figure 1. Illustration of patc h window and sear ch window. l and h are the position of two pixel points in the image, but h is only fixed at the search window in which l is the center point. In addition, the two patch windows in which l and h are the center points respectively are used to compute the weight wl,h. Then the discrete iterative scheme of can be ob tained by 1k l u 1, , 11 ,, ,,,, 12 2 kk lhlh h Dllh h kkk k Dlll hhll hl hhll h hh uuw w fvvwbbw (19) Now considering the equation (14), we can obtain v^{k+1} as follows. 111 2 1 11 11 ,0 k i kkk iNLii nk i i kk NL ii kk NL ii vxdx vMax ub vxdx ub ub (20) Obviously, (20) is not the exact generalized soft thresholding formulas as one should expect. In order to calculate it effectively, we propose the above approxi mate formulations to simplify implementation of (14) and speed up computation. Although (20) is not the ana lytic solution for and may cause a little error, the Bregman iterations may correct it automatically. This issue is confirmed in our experiments in section 4 Figure 6. By using the same manner as (19), (20) can be also rewritten in a discrete version as below. v 1 , 11 ,, 1 2, 1 ,0 k kil kk i il ilk nkil i i A B vMaxA A B (21) where, 111 ,,,, kkkk ilihillh ilh h Auuwb 1 ,, (22.1) 2 ,, k ii lh Bvk lh (22.2) Now it is time for us to give the NLCTV algorithm in detail. NLCTV Algorithm 1. Initialization: 0k ,00 0 ii bv ,; for i = 1, …, n; 0 ii uf0 k 2. Repeat 3. Update each weight by (18); i w 4. Compute each 1; kkk iiNLii bb uv 5. Compute each 1k i u from (19); 6. Compute each 1k i v from (21); 7. k=k+1; 8. Until a stopping criterion is satisfied. In above NLCTV Algorithm, the stopping criterion is usually chosen as1kkk EEE , where is the energy in the proposed model and E is a very small tolerance parameter. 3. Numerical Experiments and Analysis In this section, we will present several numerical ex periments to show the effectiveness and performance of the NLCTV model proposed in this paper for color tex ture image inpainting in terms of vision and peak signal to noise ratio (PSNR). PSNR is defined as in (23), and all experiments are performed u sing the Matlab 2010 b on a Windows 7 platform with an Intel Core 2 Duo CPU at 2.33 GHz and 2GB memory. 2 10 2 2 1 10log   n ii i nMN MAX PSNR fu (23) where stands for the layers of the color image. n and are respective the height and width of the origi nal image. AX is 255. is the restored image, and u is the original image. Copyright © 2013 SciRes. JSIP
Color Texture Image Inpainting Using the Non Local CTV Model 48 (a) (b) (c) (d) (e) Figure 2. Original image. (a) Color chess board Image. (b) Color cloth Image. (c) Color bar image.(d) River of Wisdom Image. (e) Monalisa Image. We first present the original images in Figure 2 and we can make visual comparisons with the inpainted im ages subsequently and then compute their exact PSNRs. Figure 2 (a) is a synthetic color texture image and Fig ure 2(b) is real color texture image. These two images are set up for testing the capability of our proposed model for color texture image inpainting . Figure 2(c) is a color bar image presented here to observe the edge pre serving phenomenon. The last two images are two real famous images blended with texture and nontextur e, and we will use them to further demonstrate their inpainting results in the subsequent experiments. (a) (b) (c) (d) (e) (f) Figure 3. Color chess board inpainting. (a) Image with mask marked by black square; (b)(c) Intermediate results by proposed NLCTV model; (d) Final result by proposed NLCTV model; (e) Final result by TV inpainting method [4]; (f) Final result by elastica inpainting method [11]. (a) (b) (c) (d) (e) (f) Figure 4. Color cloth inpainting. (a) Image with mask marked by white rectangle; (b)(c) Intermediate results by proposed NLCTV model; (d) Final result by proposed NLCTV model; (e) Final result by TV inpainting method [4]; (f) Final result by elastica inpainting method [11]. In the first experiment as shown in Figure 3, we aim to restore a toy image with regular grids and a large black broken region. Here we also show the inpainting results of TV model proposed in [4] and elastica inpainting model reported in [11]. The advantages of TV inpainting model are its simple manipulation and fast computation, but the major drawback of this model is that it does not restore satisfactorily a single object when the discon nected remaining parts are separated far apart by the in painting domain. One can observe in Figure 3(e) that one cannot get a desirable result when dealing with the nontexture images of large broken domain. In order to overcome this problem, the elastica in painting model [11] is subsequently tested. Remind that this is a model only suitable for nontexture images with large broken do main, and the resu lt is shown in Figure 3(f). In fact, the model in [11] has meaningful statistical fluctuations in textures and the textures are often smoothed out by its PDEs. In conclusion, we notice that the proposed NLCTV using the non local information can obtain a perfect result as shown in Figure 3(d) in comparison with the original image in Figure 2(a). In fact, the TV inpainting model [4] and elastica inpainting model [11] cannot find the changes of the texture and thus could not recover this large broken texture region as shown Figures 3(e) (f). In order to validate our proposed model with the pro posed fast algorithm on real texture images and demon strate a further illustration, we now set up the second experiment properly. In this experiment, the regions of missing data are quite large with respect to textures as shown in Figure 4 (size is). The size of missing region in Figure 4(a) is above and 81 81 10 23846 be low, and that means the proportion of total texture miss ing part is ab out 10% of the original image. Here, we use 41 41 of search window to inpaint the missing region. In this case, the proposed NLCTV model can obtain Copyright © 2013 SciRes. JSIP
Color Texture Image Inpainting Using the Non Local CTV Model 49 perfect results while the other two approaches failed as shown in Figure 4. Next we will show that choosing a suitable search window for our model is a very crucial issue. In Figure 5, several approximations of Figure 4(a) have been calculated with different sizes of search windows and they are listed for comparison with each other. When the s izes of search w indows are 11 11 and , the inpainting results are obviously not satisfactory as shown in Figures 5(a) and (b). So two relative larger ones such as sizes of and 21 21 31 314141 are used further and we can see better visual results in Figures 5(c) and (d). However, the final PSNR values from Table1 tell us that the best result comes from implementation with the largest search window. However, the computation time becomes much more expensive with increase of the search window size consequently as demonstrated in Table 1. In conclusion, for these diverse search windows, we find that small search window is not sufficient to the success of inpainting the large missing part of the texture image, and usually a large search window is needed, which is usually time consuming. In order to show time complexity, the time of constructing the weight function and the total computation time in the experi ments are shown in Table 1. In fact, in our experiment to obtain Figure 5, we update every 30 iterative steps in order to improve their computational efficiency, and the total iterative numbers are set to be 300. Techni cally, decreasing the updating frequency of the weight function will lead to increasing of iteration steps, and how to balance them is our future research topic. ,wxy ,wxy ,wxy Except for the visual results, we can calculate the quantity evaluations for our inpainting results in Figure 3, Figure 4 with different techniques as shown in Table 2. The results in this Table show a great success of our proposed model in inpainting both the synthetic and real images with color texture. (a) (b) (c) (d) Figure 5. Test the effect of different search windows on inpainting results. (a)(d) are results with search windows of 11 × 11, 21 × 21, 31 × 31 and 41 × 41, respectively. The NLTV model is for gray images and if we use it directly to different layers separately for color image inpainting, we find that this will lead to smear edges as demonstrated in Figure 6(b). However, considering the coupling of different layers of colo r images and using the NLCTV regularizer term in our proposed model, one can observe that the edge is perfectly preserved as shown in Figure 6(c). Moreover, Figure 6(d) presents the en ergy function with each iteration of the proposed algo rithm and it shows the convergence of the proposed algo rithm in this simulation. This exp eriment not only proves that our model does an excellent job in color image edge preserving but also shows that the Split Bregman de signed for the proposed model is convergent. Table 1. Comparisons of PSNRs and computation time us ing different search windows. Images Figure 2(a) Figure 2(b) Figure 2(c) Figure 2(d) Size of the search window 1111 21 21 31 31 4141 PSNR of the restored image using different search window 30.07632.601 36.581 40.054 Time(s) of constructing the weight function15.05447.464 96.659 149.508 Total computation time(s) 623.867 1510.226 2786.269 4666.216 Table 2. Comparisons of PSNRs using different methods. Experiments Figure 3 Figure 4 PSNR of the damaged image 11.795 17.873 PSNR of the restored image using NLCTV model 36.059 40.054 PSNR of the restored image using TV inpainting m e thod 15.147 26.665 PSNR of the restored image using elastica inpainting method 16.842 28.862 (a) (b) (c) ( d) Figure 6. Edge preserving test using different non local model and show convergence of NLCTV. (a) Damaged image; (b) Final result by NLTV directly using to different layers of color image; (c) Final result by proposed NLCTV. (d) Energy decreasing plot of NLCTV. In the next two experiments, we will compare the proposed model with the stateoftheart method pro posed in [18] both quantitatively and visually. Figure 7 presents the picture of River of Wisdom inpainting in different cases, while Figure 8 shows the picture of Monalisa inpainting. In Figure 7, We made the damage types marked by varied color paintings to illustrate that our model can adaptively inpaint such complicated con tamination effectively. Though we cannot see visual dif ferences between Figures 7(b) and (c) but the PSNR val ues acquired by the proposed method is about 4.3 higher Copyright © 2013 SciRes. JSIP
Color Texture Image Inpainting Using the Non Local CTV Model 50 than that obtained by the stateoftheart method [18]. In order to see the detailed difference, we crop a small block from the damaged images in two cases as shown in Figures 7(e)(f) and Figures 8(e)(f), we can see that th e proposed NLCTV performs better in micro structures for the poles of the ship in Figure 7(f). Similarly, the visual effect and PSNR value show the benefit of the proposed model and the micro structures in hand show the edge persevering differences in Figures 8(e)(f). These experiments demonstrate the advantages of the proposed model in color texture image inpainting. In the above experiments, the damaged areas D are different one can see that in all cases, the proposed model works well with extraordinary performance. Also the types of damages are different. (a) (b) (c) (d) (e) (f) Figure 7. The River of Wisdom inpainting. (a) Damaged image; (b) Final result by [18], PSNR = 31.58; (c) Final re sult by proposed NLCTV model, PSNR = 35.89; (d)(f) Zoomed small subregions (indicated by black rectangle in (a)) of the images in (a)(c) for detail comparison. (a) (b) (c) (d) (e) (f) Figure 8. Monalisa inpainting. (a) Damaged image; (b) final result by method in [18], PSNR = 36.15; (c) final result by proposed NLCTV model, PSNR = 39.85; (d)(f) zoomed small subregions (indicated by black rectangle in (a)) of the images in (a)(c) for detail comparison. 4. Conclusions In this paper, by using the relevant concepts of non local operators and the CTV model, we proposed the NLCTV model for color texture image inpainting as an extension of CTV model for color image denoising. In this model, the mask is automatically assigned. Then we design a new Split Bregman algorithm and provide their implementations in detail. Numerical experiments validate the performance of the proposed model for color texture image inpainting in different cases. REFERENCES [1] G. Aubert. and P. Kornprobst, “Mathematical Problems in Image Processing: Partial Differential Equations and the Calculus of Variations 2nd ed. Berlin, Germany: SpringerVerlag, 2006. [2] M. Bertalmío, G. Sapiro, V. Caselles and C. Ballester, “Image Inpainting,” In Proceedings of SIGGRAPH, 2000, pp. 417424. doi:10.1145/344779.344972 [3] M. Bertalmio, A. Bertozzi and G. Sapiro, “NavierStokes, FluidDynamics, and Image and Video Inpainting,” In Proc. of IEEECVPR, 2001, pp. 355362. [4] T. Chan and J. Shen, “Mathematical Models of Local Nontexture Inpaintings,” SIAM Journal of Applied Mathematics, Vol. 62, No. 3, 2002, pp. 10191043. doi:10.1137/S0036139900368844 [5] L. Rudin, S. Osher and E. Fatemi, “Nonlinear Total Variation Based Noise Removal Algorithms,” Physica D:Nonlinear Phenomena, Vol. 60, No. 1/4, 1992, pp. 259268. doi:10.1016/01672789(92)90242F [6] A. Bertozzi, S. Esedoglu and A. Gillette, “Inpainting of Binary Images Using the CahnHilliard Equation,” IEEE Trans. Image Processing, Vol. 16, No. 1, 2007, pp. 285291. doi:10.1109/TIP.2006.887728 [7] M. Nitzberg, D. Mumford and T. Shiota, Filering, seg mentation, and depth, LNCS, Vol. 662, Berlin, Germany: SpringerVerlag, 1993. http://dx.doi.org/10.1007/3540564845 [8] T. Chan and J. Shen, “Nontexture Inpainting by Curva turedriven Diffusions (CDD),” Journal of Visual Com munication and Image Representation., Vol. 12, No. 4, 2001, pp. 436449. doi:10.1006/jvci.2001.0487 [9] T. Chan, S. Kang and J. Shen, “Euler's Elastica and Cur vaturebased Image Inpainting,” SIAM J. Appl. Math., Vol. 63, No. 2, 2002, pp. 564592. [10] S. Esedoglu and J. Shen, “Digital Inpainting Based on the MumfordShahEuler Image Model,” European Journal of Applied Mathematics, Vol. 13, No. 4, 2002, pp. 353370. doi:10.1017/S0956792502004904 [11] X. Tai, J. Hahn and G. Chung, “A Fast Algorithm for Euler’s Elastica Model Using Augmented Lagrangian Method,” SIAM J. Appl. Math., Vol. 4, No. 1, 2011, pp. 313344. [12] A. Efros and T. Leung, “Texture Synthesis by Nonpara Metric Sampling,” In Proc. of the IEEE ICCV, 1999, pp. 10331038. Copyright © 2013 SciRes. JSIP
Color Texture Image Inpainting Using the Non Local CTV Model Copyright © 2013 SciRes. JSIP 51 [13] A. Criminisi, P. Pérez, and K. Toyama, “Region Filling and Object Removal by Exemplarbased Inpainting,” IEEE Transactions on Image Processing, Vol. 13, No. 9, 2004, pp. 12001212. doi:10.1109/TIP.2004.833105 [14] Z. Xu and S. Jian, “Image Inpainting by Patch Propaga tion Using Patch Sparsity,” IEEE Transactions on Image Processing, Vol. 19, No. 3, 2010, pp. 11531165. [15] P. Arias, V. Caselles, G. Facciolo and G. Sapiro, “A Variational Framework for Exemplarbased Image In painting,” International Journal of Computer Vision, Vol. 93, No. 3, 2011, pp. 129. doi:10.1007/s1126301004187 [16] P. Arias, V. Caselles and G. Facciolo, “Analysis of a Variational Framework for ExemplarBased Image In painting,” SIAM Multiscale Modeing Simulation, Vol. 10, No. 2, 2012, pp. 473514. doi:10.1137/110848281 [17] M. Bertalmio, L. Vese, G. Sapiro and S. Osher, “Simul taneous Structure and Texture Image Inpainting,” IEEE Transactions Image Processing, Vol. 12, No. 8, 2003, pp. 882889. doi:10.1109/TIP.2003.815261 [18] J. Cai, R. Chan and Z. Shen, “A Frameletbased Image Inpainting Algorithm,” Applied and Computational Har monic Analysis, Vol. 24, No. 2, 2008, pp. 131149. doi:10.1016/j.acha.2007.10.002 [19] J. Cai, R. Chan and Z. Shen, “Simultaneous Cartoon and Texture Inpainting,” Inverse Problems and Imaging, Vol. 4, No. 3, 2010, pp. 379395. doi:10.3934/ipi.2010.4.379 [20] A. Buades, B. Coll and J. Morel, “A Review of Image Denoising Algorithms, with a New One,” SIAM Multis cale Modeling Simulation, Vol. 4, No. 2, 2005, pp. 490530. doi:10.1137/040616024 [21] G. Gilboa and S. Osher, “Nonlocal Operators with Ap plications to Image Processing,” SIAM Multiscale Mod eling Simulation, Vol. 7, No. 3, 2008, pp. 10051028. doi:10.1137/070698592 [22] G. Peyré, S. Bougleux and L. Cohen, “Nonlocal Regu larization of Inverse Problems,” LNCS, Vol. 5304, 2008, pp. 5768. [23] Y. Lou, X. Zhang, S. Osher and A. Bertozzi, “Image re covery via non local operators,” Journal of Science Computer, Vol. 42, No. 2, 2010, pp. 185197. [24] X. Zhang, M. Burger, X. Bresson and S. Osher, “Breg manized Nonlocal Regularization for Deconvolution and Sparse Reconstruction,” SIAM Journal Imag. Science, Vol. 3, No. 3, 2010, pp. 253276. [25] W. Ma and S. Osher, “A TV Bregman iterative model of Retinex theory,” Univ. California, Los Angeles, UCLA CAM Rep. 1013, 2010. [26] D. Zosso, G. Tran and S. Osher, “A Unifying Retinex Model Based on Nonlocal Differential Operators,” Univ. California, Los Angeles, UCLA CAM Rep. 1303, 2013. [27] M. Jung, X. Bresson, T. F. Chan and L. A. Vese. “Nonlocal MumfordShah Regularizers for Color Image Restoration,” IEEE Transactions on Image Processing, Vol. 20, No. 6, 2011, pp. 15831598. doi:10.1109/TIP.2010.2092433 [28] D. Mumford and J. Shah, “Optimal Approximations by Piecewise Smooth Functions and Associated Variational Problems,” CPAM, Vol. XLII, 1989, pp. 577685. [29] P. Blomgren and T. Chan, “Color TV: Total Variation Methods for Restoration of VectorValued Images,” IEEE Transactions on Image Processing, Vol. 7, No. 3, 1998, pp. 304309. doi:10.1109/83.661180 [30] J. Yang, W. Yin, Y. Zhang and Y. Wang, “A Fast Algo rithm for EdgePreserving Variational Multichannel Im age Restoration,” SIAM Journal on Imaging Sciences, Vol. 2, No. 2, 2009, pp. 569592. doi:10.1137/080730421 [31] Y. Yu, Z. Pan, W. Wei and J. Jiang, “Edge Preserving of Some Variational Models for Vectorial Image Denois ing,” Jounal of Graphics and Images, Vol. 16, No. 12, 2011, pp. 22232230. [32] T. Goldstein and S. Osher, “The Split Bregman Algo rithm for L1 Regularized Problems,” SIAM Journal on Imaging Sciences, Vol. 2, No. 2, 2009, pp. 323343. doi:10.1137/080725891 [33] Q. Wang, Z. Pa n, W. Wei, Z. Zhang, and C. Wang , “The Generalized MTVL1 and its Split Bregman Algorithm for Noise Removal of Color Images with Textures,” in The Chn. Conf. Pattern Recognition (CCPR). 2010, pp. 16.
