The phenomenon of phase transition in constraint satisfaction problems (CSPs) plays a crucial role in the field of artificial intelligence and computational complexity theory. In this paper, we propose a new random CSP called d-p-RB model, which is a generalization of RB model on domain size d and constraint tightness p. In this model, the variable domain size d Ε [ nα, nny], and all constraints are uniformly divided into several groups with different constraint tightness p. It is proved by the second moment method that the d-p-RB model undergoes phase transition from a region where almost all instances are satisfiable to a region where almost all instances are unsatisfiable as the control parameter increases. Moreover, the threshold value at which the phase transition occurs is located exactly.
The constraint satisfaction problem (CSP in short), originated from the artificial intelligence, has become an important topic in the interdisciplinary research of computer science, mathematics and statistical physics. Many problems in the fields of artificial intelligence, computer science and automatic control can be modeled as constraint satisfaction problems. Moreover, CSPs are widely used in many practical problems such as resource allocation, pattern recognition, logistics scheduling and temporal reasoning.
In general, CSP is defined on a set of variables and a set of constraints. Each variable has a corresponding non-empty domain, the domain size of the variable can be fixed, or vary with the number of variables. Each constraint involves a randomly selected subset of variables and a corresponding compatible assignments set to specify the allowable combinations of values of the variables in this constraint. The randomly selected constraints constitute a random CSP instance. An assignment that satisfies all the constraints simultaneously is called a solution of the CSP instance. Interestingly, experimental results suggest that the probability of a random CSP instance having a solution exhibits a phase transition behavior. In a seminal paper, Cheeseman et al. showed empirically that the hardest instances of CSPs often occur around a rapid transition in solubility [
In this paper, we propose a new random CSP model, called d-p-RB model, which is a generalization of RB model on constraint tightness p and the variable domain size d. In RB model, the domain size d = n α (α is a constant) is a power function of the number of variables n, and the constraint tightness p is fixed. In d-p-RB model, we uniformly divided the random constraints into several groups and diversify the domain size d as well as the constraint tightness p of the constraints in different groups. More specifically, for an instance with n variables in d-p-RB model, the domain size d ∈ [ n α , n n γ ] (α, γ are constants) is defined within a certain range rather than a single value as in RB model, for the ith group of constraints, it has its own constraint tightness p i ( 0 < p i < 1 ), which is distinct from the unchangeable p in RB model. By the second moment method, we show that the d-p-RB model can exhibit exact phase transition phenomenon under certain conditions, and the transition point can also be obtained pricey. Moreover, since both d and p are varied in d-p-RB model, it has more extensive practical significance and theoretical value.
A CSP instance I = ( U , D , C ) of d-p-RB model is defined as follows:
1) U = { u 1 , u 2 , ⋯ , u n } is a set of n variables.
2) D = { D 1 , D 2 , ⋯ , D n } is a domain set. Each variable u i ( i = 1 , 2 , ⋯ , n ) takes values from D i , whose size | D i | = d ∈ [ n α , n n λ ] , where α and γ are constants.
3) C = { C 1 , C 2 , ⋯ , C t } is a set of constraints, and each constraint C i is a pair ( S i , R i ), where S i = { s i 1 , s i 2 , ⋯ , s i k } (k is the length of the constraint) is a subset of U, and R i ⊆ D i 1 × D i 2 × ⋯ × D i k is a compatible assignments set.
A constraint C i is satisfied if the k-tuple of values assigned to variables in S i is contained in R i . A solution of a CSP instance is an assignment to all the variables that satisfies all constraints.
A random CSP instance in d-p-RB model is generated in the following two steps:
Step 1. We select with repetition l groups of constraints. For each group, there are t / l constraints with each contains k variables, which are randomly select from u, and distinct from each other.
Step 2. For each group of constraints, we uniformly select at random without repetition p i d k ( 0 < p i < 1 is the constraint tightness) compatible assignments to form the compatible assignments set R i ( i = 1 , 2 , ⋯ , l ).
We let t = r n ln d , where r is a constant control parameter, which determines how many constraints are in a CSP instance. Let p = min { p 1 , p 2 , ⋯ , p l } , where p i ( i = 1 , 2 , ⋯ , l ) determines how restrictive the constraints are. Let Pr ( s a t ) denote the probability of a random d-p-RB instance being satisfiable, then we have the following theorem.
Theorem Let r m = − l ∑ i = 1 l ln p i ( 0 < p i < 1 ), if the constants k, p, α, γ satisfy the relations α > γ + 1 k − 1 , k ≥ max { 1 p , γ + 2 } then
lim n → ∞ Pr ( s a t ) = { 0 1 when r > r m when r < r m ( 1 ) (2)
The theorem shows that, when the number of variables n is sufficiently large, there exists a sudden shift in r m .
Let N denote the number of solutions of a random CSP instance I. The expectation and the second moment of N is denoted by E ( N ) and E ( N 2 ) . When r > r m , we consider the Markov inequality Pr ( s a t ) ≤ E ( N ) . When r < r m , by
the second moment method, we estimate the upper bound of E ( N 2 ) E 2 ( N ) , and then by the Cauchy inequality Pr ( s a t ) ≥ E 2 ( N ) E ( N 2 ) , we finally attain our goal. Now we demonstrate the two cases respectively.
Since the constraints are generated independently in d-p-RB model, the expected number of solutions E ( N ) is given by
E ( N ) = d n ( p 1 p 2 ⋯ p l ) t l = exp ( n ln d + r l n ln d ∑ i = 1 l ln p i ) = exp [ n ln d ( 1 + r l ∑ i = 1 l ln p i ) ] . (3)
Since r > r m , we have
1 + r l ∑ i = 1 l ln p i < 0 , (4)
thus
lim n → ∞ E ( N ) = 0 . (5)
Then using the Markov inequality Pr ( s a t ) ≤ E ( N ) , by (5), it’s not hard to have
lim n → ∞ Pr ( S a t ) = 0 . (6)
Definition 1 (The assignment pair) Suppose that the assignment pair 〈 t i , t j 〉 is an ordered pair, where t i = ( a i 1 , a i 2 , ⋯ , a i n ) , t j = ( a j 1 , a j 2 , ⋯ , a j n ) , and a i h , a j h ∈ D h ( h = 1 , 2 , ⋯ , n ). An assignment pair 〈 t i , t j 〉 satisfies a CSP instance if and only if both t i and t j satisfy the instance.
Definition 2 (The similarity number) Define a function as follows
S a m ( a i h , a j h ) = { 1 a i h = a j h 0 a i h ≠ a j h
Assume m = ∑ h = 1 n s a m ( a i h , a j h ) , thus the assignment pair 〈 t i , t j 〉 has m
identical assignments, i.e. the similarity number of 〈 t i , t j 〉 is m. It is obvious that 0 ≤ m ≤ n .
Next, we use the second moment method to complete the proof.
Assume that P ( 〈 t i , t j 〉 ) represents the probability that t i and t j satisfy the instant I simultaneously. We analyse this probability in the following way:
Since there are m identical assignments in t i and t j , for each constraint, we have the following two cases:
1) The assignments of k variables that the constraint restricts are all same in t i and t j , in this case, the probability of 〈 t i , t j 〉 satisfying the constraint is
( p i d k − 1 d k − 1 ) ( p i d k d k ) = p i , and for a random constraint, the probability of such a situation is ( m k ) ( n k ) .
2) Otherwise, the probability of 〈 t i , t j 〉 satisfying the constraint is
( p i d k − 2 d k − 2 ) ( p i d k d k ) = p i p i d k − 1 d k − 1 , and the probability that 〈 t i , t j 〉 falls into such a situation is 1 − ( m k ) ( n k ) .
Let
σ m , n = ( m k ) ( n k ) , s = m n . (7)
Since
σ m , n = ( m k ) ( n k ) = m ( m − 1 ) ⋯ ( m − k + 1 ) n ( n − 1 ) ⋯ ( n − k + 1 ) ≤ ( m n ) k (8)
we have
σ m , n ≤ s k . (9)
Since the constraints are generated independently, the assignment pair 〈 t i , t j 〉 satisfying all the constraints in random instance i is
P ( 〈 t i , t j 〉 ) = ∏ i = 1 l [ p i σ m , n + p i p i d k − 1 d k − 1 ( 1 − σ m , n ) ] t l ≤ ∏ i = 1 l p i t l [ σ S , n + p i ( 1 − σ S , n ) ] t l ≤ ∏ i= 1 l p i t l [ p i + ( 1 − p i ) s k ] t l ≤ ∏ i = 1 l p i 2 t l ( 1 + 1 − p i p i s k ) t l . (10)
Let A m be the set of assignment pairs whose similarity number is m, | A m | be the cardinality of A m , then we have
| A m | = d n ( n m ) ( d − 1 ) n − m = d 2 n ( n m ) ( 1 − 1 d ) n − m ( 1 d ) m . (11)
Thus by (10) and (11), the second order moment of the number of solutions of the random instance of d-p-RB model is
E ( N 2 ) = ∑ m = 0 n | A m | P ( 〈 t i , t j 〉 ) ≤ ∑ m = 0 n d 2 n ( n m ) ( 1 − 1 d ) n − m ( 1 d ) m ∏ i = 1 l p i 2 t l ( 1 + 1 − p i p i s k ) t = E 2 ( N ) ∑ m = 0 n ( n m ) ( 1 − 1 d ) n − m ( 1 d ) m ∏ i = 1 l ( 1 + 1 − p i p i s k ) t l = E 2 ( N ) ∑ 0 ≤ s ≤ 1 B ( s ) W ( s ) , (12)
where
B ( s ) = ( n s n ) ( 1 − 1 d ) n − n s ( 1 d ) n s (13)
W ( s ) = ∏ i = 1 l ( 1 + 1 − p i p i s k ) t l , (14)
i.e.,
E ( N 2 ) E 2 ( N ) ≤ ∑ 0 ≤ s ≤ 1 B ( s ) W ( s ) . (15)
Considering that 0 ≤ s ≤ 1 , in order to evaluate the upper bound of the above Inequality (15), we divide the interval [0,1] into three parts: [ 0 , s 1 ] , [ s 1 , s 2 ] ,
[ s 2 , 1 ] , where s 1 = 1 n β , s 2 = 1 n γ + 1 k − 1 , here β and γ satisfy γ + 1 k − 1 < β < min { 1 , α } .
1) For s ∈ [ 0 , s 1 ] , let s 1 = 1 n β , where γ + 1 k − 1 < β < min { 1 , α } , recalling that
t = r n ln d , d ∈ [ n α , n n γ ] , and p = min { p 1 , p 2 , ⋯ , p l } we have
W ( s ) = ∏ i = 1 l ( 1 + 1 − p i p i s k ) r n ln d l ≤ ( 1 + 1 − p p s k ) r n ln d ≤ exp [ r n ln d ln ( 1 + 1 − p p s k ) ] ≤ exp ( r n ln d ⋅ 1 − p p ⋅ s k ) ≤ exp [ r ( 1 − p ) p n γ + 1 − β k ln n ] . (16)
Since β > γ + 1 k − 1 > γ + 1 k , γ + 1 − β k < 0 , we get
lim n → ∞ W ( s ) = 1 . (17)
Then it is not hard to obtain that
∑ 0 ≤ s ≤ s 1 B ( s ) W ( s ) ≤ ∑ 0 ≤ s ≤ s 1 B ( s ) ≤ ∑ 0 ≤ s ≤ 1 B ( s ) = 1 . (18)
2) For s ∈ [ s 1 , s 2 ] , let s 2 = 1 n γ + 1 k − 1 , f ( s ) = − s ln s − ( 1 − s ) ln ( 1 − s ) , by the Stirling’s formula n ! = ( n e ) n 2 π n e ε 12 n , where | ε | < 1 , it’s not hard to see
( n n s ) < e n h ( s ) . (19)
Since
B ( s ) W ( s ) ≤ ( n s n ) ( 1 − 1 d ) n − n s ( 1 d ) n s ( 1 + 1 − p p s k ) r n ln d (20)
we get
ln B ( s ) W ( s ) ≤ n f ( s ) + ( n − n s ) ln ( 1 − 1 d ) − n s ln d + r l n ln d ln ( 1 + 1 − p p s k ) ≤ n [ f ( s ) − s ln d + r ( 1 − p ) l p ⋅ ln d ⋅ s k ] ≤ n s [ − ln s − 1 − s s ln ( 1 − s ) − α ln n + r ( 1 − p ) l p ⋅ n γ ⋅ ln n ⋅ s k − 1 ] . (21)
For s ∈ [ s 1 , s 2 ] , we have
ln B ( s ) W ( s ) ≤ n s [ β ln n + 1 − α ln n + r ( 1 − p ) l p ⋅ n γ ⋅ ln n ⋅ ( 1 n γ + 1 k − 1 ) k − 1 ] = n s ln n [ β − α + 1 ln n + r ( 1 − p ) l p ⋅ 1 n ] ≤ n 1 − β ln n [ β − α + o ( 1 ln n ) ] . (22)
Since β < min { 1 , α } , we have
lim n → ∞ ln B ( s ) W ( s ) = − ∞ , (23)
hence
∑ s 1 ≤ s ≤ s 2 B ( s ) W ( s ) ≤ n B ( s ) W ( s ) ≤ n exp [ n 1 − β ln n ( β − α + o ( 1 ln n ) ) ] = n n 1 − β ( β − α + o ( 1 ln n ) ) + 1 → 0 ( n → ∞ ) . (24)
Thus, for arbitrary small ε, there exists an integer N 1 > 0 , such that
∑ s 1 ≤ s ≤ s 2 B ( s ) W ( s ) < ε 2 . (25)
3) For s ∈ [ s 2 , 1 ] we have
B ( s ) W ( s ) = ( n s n ) ( 1 − 1 d ) n − n s ( 1 d ) n s ∏ i = 1 l ( 1 + 1 − p i p i s k ) r n ln d l (26)
Then
ln B ( s ) W ( s ) ≤ n f ( s ) − n s ln d + r l n ln d ∑ i = 1 l ln ( 1 + 1 − p i p i s k ) = n [ f ( s ) + ln d ( − s + r l ∑ i = 1 l ln ( 1 + 1 − p i p i s k ) ) ] . (27)
Let g ( s ) = r l ∑ i = 1 l ln ( 1 + 1 − p i p i s k ) − s , differentiating g ( s ) with respect to s, we get
g ′ ( s ) = r l ∑ i = 1 l k ( 1 − p i ) s k − 1 p i + ( 1 − p i ) s k − 1 , (28)
and then
g ″ ( s ) = r l ∑ i = 1 l k ( 1 − p i ) s k − 2 [ ( k − 1 ) p i + ( 1 − p i ) s k ] [ p i + ( 1 − p i ) s k ] 2 . (29)
By the condition k ≥ 1 p , we have g ″ ( s ) ≥ 0 , which implies that g ( s ) is convex for s ∈ [ 0 , 1 ] . Note that g ( 0 ) = 0 and g ( 1 ) = − r l ∑ i = 1 l ln p i − 1 < 0 for r < r m = − l / ∑ i = 1 l ln p i , therefore we get g ( s ) < 0 for s ∈ [ s 2 , 1 ] . Let max s ∈ [ s 2 , 1 ] g ( s ) = − M , where M > 0 is a constant.
For f ( s ) = − s ln s − ( 1 − s ) ln ( 1 − s ) , similarly we have
f ′ ( s ) = − ln s + ln ( 1 − s ) (30)
f ″ ( s ) = − 1 s ( 1 − s ) < 0 . (31)
So f ( s ) is concave and has the maximum value ln 2 at s = 1 2 . Thus we have
ln B ( s ) W ( s ) ≤ n [ f ( s ) + ln d ⋅ g ( s ) ] ≤ n ( ln 2 − M ln d ) . (32)
So we get
∑ s 2 ≤ s ≤ 1 B ( s ) W ( s ) ≤ n B ( s ) W ( s ) ≤ n exp ( n ln 2 − M ln d ) ≤ 2 n n − α M n + 1 , (33)
hence
lim n → ∞ ∑ s 2 ≤ s ≤ 1 B ( s ) W ( s ) = 0 , (34)
i.e., for s ∈ [ s 2 , 1 ] , there exists an integer N 2 > 0 , such that
∑ s 2 ≤ s ≤ 1 B ( s ) W ( s ) < ε 2 . (35)
Summarizing the above, from (18), (25), (35), letting N = max { N 1 , N 2 } , we obtain
∑ 0 ≤ s ≤ 1 B ( s ) W ( s ) < 1 + ε . (36)
Thus we have
E ( N 2 ) E 2 ( N ) ≤ 1 + ε , (37)
then by the Cauchy inequality Pr ( s a t ) ≥ E 2 ( N ) E ( N 2 ) , we have
1 1 + ε ≤ Pr ( s a t ) ≤ 1 , (38)
then we get
lim n → ∞ Pr ( s a t ) = 1 . (39)
Thus the theorem is proved.
So far we have demonstrated the satisfiability phase transition in theory. From the proof of the theorem, it can be seen that when the control parameter r is less than the transition point r m , the probability of a CSP instance being satisfied tends to 1, while the control parameter r is greater than the transition point r m , the probability tends to 0. Thus there exists a sharp threshold in the CSP instances generated by d-p-RB model.
In this paper, we propose a new CSP model d-p-RB. Compared with RB model, we diversify the constraint tightness p and broaden the domain size d. By the method of second moment, we proved that there indeed exist satisfiability phase transition phenomenon and the transition point can also be located exactly.
Liu, Y.N. (2017) Sharp Thresholds for a Random Constraint Satisfaction Problem. Open Journal of Applied Sciences, 7, 574-584. https://doi.org/10.4236/ojapps.2017.710041