Open Journal of Stomatology
Vol.3 No.1(2013), Article ID:28733,6 pages DOI:10.4236/ojst.2013.31004

Effects of hyaluronan oligosaccharides on apoptosis of human gingival fibroblasts

Yuki Tanne, Kotaro Tanimoto, Satoru Okuma, Ryo Kunimatsu, Naoto Hirose, Tomomi Mitsuyoshi, Kazuo Tanne

Department of Orthodontics, Applied Life Sciences, Hiroshima University Institute of Biomedical & Health Sciences, Hiroshima, Japan


Received 22 January 2013; revised 25 February 2013; accepted 1 March 2013

Keywords: Hyaluronan Oligosaccharide; Apoptosis; Caspase-3


An increased content of low molecular weight (LMW)- HA was found in human gingiva during the initial phase of periodontitis. A number of functions have been described for LMW-HA in inflammatory processes. It is also demonstrated that LMW-HA induces apoptosis in chondrocytes and tumor cells, however, effects of LMW-HA on the apoptosis of gingival fibroblasts (GFs) remain unclear. The purpose of this study was to investigate the effects of HA oligosaccharides (HAoligo) on the apoptosis of GFs. GFs were isolated from palatal gingival tissues around the upper first premolars of eight youth (6 - 10 years old) and eight adult (26 - 35 years old) subjects. Cultured GFs were subjected to the analyses, ELISA for cell proliferation, quantitative real time-PCR for gene expression of Bcl-2, and caspase-3 assay for apoptotic potency. Proliferative activity of cultured GFs was increased significantly by the treatment with HAoligo in both young and adult groups. HAoligo treatment also increased caspase-3 activity and decreased Bcl-2 gene expression in cultured GFs, whereas the treatment with high molecular weight—HA exerted no influences on caspase-3 activity in both groups. From these results, it is suggested that HAoligo play a crucial role in the induction of apoptosis in GFs, leading to the destruction of gingival tissues.


Hyaluronan (HA) is a non-sulfated high-molecular-mass glycosaminoglycan (GAG) composed of repeating disaccharide residues of N-acetylglucosamine and glucuronic acid and distributes throughout the extracellular space of connective tissues. HA is one of major extracellular matrix components in gingival tissues and critical for the structural organization [1,2].

HA exerts different biological effects on cells and tissues in response to its molecular weight. An increased content of low molecular weight (LMW)-HA was found in human gingiva during the initial stage of periodontitis [3] and chronic inflammation [4]. A number of functions have been described for LMW-HA in inflammatory processes [5,6]. The expression of MMP-1 in cultured HPDL cells was accelerated by HAoligo treatment [7]. From these findings, it is speculated that LMW-HA may induce degeneration or destruction of gingival tissues.

Previous studies suggested that tissue destruction by apoptosis played a relevant role in the pathogenesis of chronic adult periodontitis [8,9]. However, apoptosis is a highly controlled process which is affected by various factors in destructive process [10]. There are various apoptotic signal transduction pathways, in which major one is through death receptors and mitochondria. Mitochondria play an essential role in many forms of apoptosis by releasing apoptogenic factors such as cytochrome c [11] and apoptosis-inducing factor (AIF) [12] which activates death proteases called caspases [13]. Active effector caspases such as caspase-3, -6 and -7 mediate the cleavage of an overlapping set of protein substrates, resulting in morphological features of apoptosis [14]. Meanwhile, Bcl-2 is known as a member of the anti-apoptotic proteins and involved in the inhibition of apoptosis [15].

A previous study reported that injection of high molecular weight (HMW)-HA decreased caspase-3, 9 expression and the number of Fas-positive cells in rat osteoarthritis model [16]. On the other hand, the treatment with HAoligo (3 - 10 disaccharide units) enhanced caspase-3 activity in tumor cells [17]. From these findings, it may be assumed that HA regulates apoptosis in chondrocytes and tumor cells, and the effects depend on its molecular weight. However, effects of LMW-HA on the apoptosis of GFs remain unclear. Thus, it is hypothesized that LMW-HA may play a crucial role in the induction of apoptosis in GFs, leading to the destruction of gingival tissues.

In this study, we investigated the effects of HAoligo on the apoptosis of human GFs, and compared the effects between young and adult subjects.


2.1. Cell Culture

Permission for a series of experiments in this study was granted by the Ethics Committee of Hiroshima University. Informed consent was obtained from sixteen patients prior to a series of experiments. GFs were isolated from palatal gingival tissues around the upper first premolars of young (6 - 10 years old) and adult (26 - 35 years old) subjects. After surgically excised, gingival tissue was subsequently minced and GFs were seeded on 100 mm dishes (Corning, New York, NY, USA) covered with 0.2% type I collagen gel (Koken, Tokyo, Japan). The explants were cultured in Dulbecco’s Modified Eagle Medium (DMEM; Sigma Aldrich, St. Louis, MO, USA) containing 10% fetal bovine serum (FBS; Daiich Chemical, Tokyo, Japan), 2.5 g/ml NaHCO3 (Katayama Chemical, Osaka, Japan), 0.7% L-glutamine (Sigma Aldrich), 120 μg/ml kanamycin (Meiji-Seika, Tokyo, Japan). The cultures were incubated under humidified conditions with 95% air and 5% CO2 at 37˚C. The medium was changed every other day. When the cells growing out from the explants became confluent, the explants were removed, and the monolayers were treated with phosphate buffer saline (PBS) containing 0.2% trypsin (Nacalai Tesque, Kyoto, Japan) and 0.1% EDTA (Wako Pure Chemical Industries, Osaka, Japan). The cells were separated well by pipetting, seeded on 100 mm culture dish (Corning) and cultivated to confluence in medium containing 10% FBS (Daiichi Chemical). The 5th passaged cells from sixteen different preparations isolated from sixteen different donors were used for the following experiments.

2.2. Treatment with HMW-HA or HAoligo

HAoligo (1.2 × 103 Da) was purchased from Cosmo Bio Corporation (Tokyo, Japan). In this experiment, we used HA hexasaccharide (HA6) as HAoligo, which is the smallest fragment of HA capable of binding to cell surface receptors [18].

GFs were seeded at a density of 3 × 104 cells/well onto 35 mm culture dish and cultured to 100% confluence. Serum deprivation was performed along a gradient from 5 and 1 to 0%, 12 h before treatment. These cells were treated with 0.1 and 1.0 mg/ml HMW-HA (2 × 106 Da Suvenyl from Chugai Pharmaceutical Co., Tokyo, Japan) and 20 and 200 μg/ml HAoligo (Cosmo Bio) for 12 h. The optimal treatment time for the induction of caspase- 3 was pre-defined with a reference to a preliminary study (data not shown).

2.3. Analysis of Cell Proliferation Activity by BrdU Immunoassay

GFs were seeded on a 96-well plate (Becton, Dickinson and Company, Franklin Lakes, NJ, USA) coated with type I collagen (Koken) at a density of 1.0 × 103 cells/ well until 60% confluence in DMEM (Sigma Aldrich) with 10% FBS (Daiichi Chemical). Cell proliferation was determined by measuring 5-bromo-2-deoxyuridine (BrdU) incorporation into the DNA of proliferating cells. The incorporation of BrdU into DNA over a time period of 24 h was detected using a Cell Proliferation ELISA BrdU kit (Roche Diagnostics, Basel, Switzerland) according to the manufacturer’s instructions. Absorbance was read at a wavelength of 570 nm using a microplate reader (Model 550, BioRad Laboratories, Hercules, CA, USA).

2.4. Quantitative Real-Time PCR Analysis

Bcl-2 is assumed to be involved in the inhibition of apoptosis, therefore, we investigated the effects of HAoligo on Bcl-2 gene expression in cultured GFs.

Total RNA was extracted from cultured GFs using a TRIzol reagent (Invitrogen, Carlsbad, CA, USA). The first-strand cDNA was synthesized from 1 μg total RNA using a ReverTra Ace-α first-strand cDNA synthesis kit (Toyobo, Osaka, Japan). Quantitative real-time PCR analysis was carried out using a Light Cycler Quick System 350S (Roche Diagnostics) under the following conditions: denaturation at 94˚C for 15 s, annealing at 55˚C for 30 s and primer extension at 72˚C for 10 s for 36 cycles. Primer sequences of Bcl-2 and glyceraldehyde 3-phosphate dehydrogenase (GAPDH) are shown in Table 1.

Thermal cycling and fluorescence detection were performed, and the quantitative results of real-time PCR were assessed with a cycle threshold [14] value, which identifies a cycle when the fluorescence of given sample becomes significantly different from the base signal. Quantification of signals was performed by normalizing their signals relative to those of GAPDH. Normalized Ct

Table 1. Oligonucleotide primers used for quantitative realtime PCR (F forward, R reverse).

GAPDH: glyceraldehyde 3-phosphate dehydrogenase.

values were expressed relative to that of Control.

2.5. Measurement of Caspase-3 Activity

Caspase-3 activity in live GFs was measured with a NucView 488 Caspase-3 Assay Kit for Live Cells (Biotium, Hayward, CA, USA) according to the manufacturer’s instructions. The NucViewTM 488 caspase-3 Substrate is a true enzyme substrate that detects caspase- 3 in live cells without interfering with the enzyme activity. Briefly, GFs were seeded at a density of 3 × 104 cells/well onto 35 mm culture dish and cultured to 100% confluence. The media were aspirated, and cells were incubated with 5 μmol/L NucViewTM 488 caspase-3 substrate in PBS for 1 h. Following the incubation, specific binding was detected by colorimetric estimation at 520 nm with a reference wavelength of 480 nm.

2.6. Statistical Analysis

All assays were performed in triplicate and reported with three different samples for each subject. Means and standard deviations were calculated from the data obtained and then subjected to Student’s t-test or one-way ANOVA followed by Scheffe’s multiple comparisons test using Statview software (Abacus Concepts Inc., San Diego, CA, USA). Resulting p-values of less than 0.05 and 0.01 were defined as significant.


3.1. Proliferative Activity of Cultured GFs in Young and Adult Groups

Proliferative activity of cultured GFs increased and reached a maximum on day 7 in both groups (Figure 1). No significant (p > 0.05) differences in proliferative activity were found between both groups on days 3 and 9, however, the proliferative activity was significantly (p < 0.05) greater in young group than in adults on days 5 and 7.

3.2. Effects of HMW-HA and HAoligo on Caspase-3 Activity in Cultured GFs

HMW-HA treatment exerted no significant (p > 0.05) effects on caspase-3 activity in both young and adult groups (Figure 2(a)). Meanwhile, treatment of HAoligo increased caspase-3 activity of GFs in a dose dependent manner in both groups. Moreover, the caspase-3 activity stimulated by HAoligo was significantly (p < 0.05) higher in young group than in adult group (Figure 2(b)).

3.3. Effects of HAoligo on Bcl-2 Gene Expression in Cultured GFs

HAoligo treatment decreased Bcl-2 expression in a dose

Figure 1. Proliferative activity of cultured GFs in young and adult groups. Cell proliferation was evaluated by measuring BrdU incorporation into the DNA of proliferating cells 3, 5, 7, and 9 days after seeding. *p < 0.05, n = 8.

Figure 2. Effects of HMW-HA and HAoligo on caspase-3 activity in cultured GFs. GFs were treated with 0.1 or 1.0 mg/ml HMW-HA, 20 or 200 μg/ml HAoligo for 12 h. The caspase-3 activity in live GFs was measured after treatment with HMWHA (a) and HAoligo (b) for 12 h. *p < 0.05, n = 8.

dependent manner, however, no significant (p > 0.05) differences in Bcl-2 expression were found between young and adult groups (Figure 3).


In this study, we investigated the effects of LMW-HA on apoptosis of GFs derived from human donors of different ages. GFs were isolated from palatal gingival tissues in young (6 - 10 years old) and adult (26 - 35 years old) subjects. Our results indicated that proliferative activity of GFs was significantly higher in young group than in adult group. In a previous study, proliferation rates of various cell strains were examined and decrease in proliferative activity was correlated with aging [19]. From these findings, it is suggested that age of donors exhibits a negative correlation with cell proliferation. Such correlation may be explained by the following findings in previous studies. The composition of glycosaminoglycan

Figure 3. Effects of HAoligo on Bcl-2 gene expression in cultured GFs. Bcl-2 mRNA levels were analyzed by real-time PCR analysis in GFs treated with 0, 20, or 200 μg/ml HAoligo for 12h. *p < 0.05, **p < 0.01, n = 8, dotted line; control level.

is different with aging. Dermatan and chondroitin sulfate decreases, but heparan sulfate increases in human fibroblasts with aging [20]. Heparan sulphate is known to play important roles in controlling cell growth [21,22]. Thus, it is suggested that increase in heparan sulfate content might be related to reduce proliferative activity with aging.

One of the earliest and most consistently observed features of apoptosis is the induction of cytosolic proteases known as caspases. Among the caspase family, caspase-3 is considered an executor enzyme, because it can be activated by several active caspases [14]. Therefore, the presence of active caspase-3 is considered as a hallmark of apoptosis activation. Our present results indicated that HAoligo treatment increased caspase-3 activity in cultured GFs. A recent study has shown that the treatment with HAoligo (3 - 10 disaccharide units) enhanced caspase-3 activity in tumor cells [17]. Furthermore, fragmented HA increased the Fas-mediated apoptosis in synovial cells derived from patients with rheumatoid arthritis [23]. In addition, HAoligo treatment decreased Bcl-2 mRNA expression in a dose dependent manner. These results suggest that HAoligo has biological activities in cultured GFs such as promotion of apoptosis by regulating antiand pro-apoptotic processes. Furthermore, the levels of caspase-3 activity were higher in young group than in adult group after treatment with HAoligo, suggesting that GFs derived from young subjects were more sensitive to HAoligo treatment and more susceptible for cell apoptosis.

In previous studies, initiation of nitric oxide (NO)·- induced apoptosis was associated with decreased Bcl-2 expression [24]. The overexpression of anti-apoptotic Bcl-2 family members suppressed NO-induced apoptosis [25]. Since HAoligo has a potential to augment the production of NO [26], the exogenous HAoligo may cause NO-induced apoptosis through down-regulation of Bcl-2 in cultured GFs. However, the induction mechanism of apoptosis by HAoligo in GFs has not been clarified, and need more extensive investigation in a future study.

In this study, the treatment of HMW-HA exerted no significant effects on caspase-3 activity in cultured GFs, although a previous study reported that injection of HMW-HA into the joint decreased the expression of caspase-3, 9 expression and the number of Fas-positive cells in rat osteoarthritis model [16]. Moreover, HMWHA significantly reduced the anti-Fas-induced apoptosis in chondrocytes from human osteoarthritis [27]. From these reports, it is assumed that HMW-HA may suppress apoptosis under pathological condition, but the effects can be varied according to the cell types.

LMW-HA distributes highly in the gingival tissues of patients at an initial phase of periodontitis [28]. The pathological accumulation of LMW-HA is suggested to be due to various degrading mechanisms such as depolymerization with reactive oxygen species [29] and aceleration of LMW-HA synthesis [30]. In addition, a number of mammalian hyaluronidases (HAase) capable of enzymatically degrading hyaluronan have recently been shown to be associated with inflammation [31]. It is thus suggested that HAoligo could be generated through HAase activity, and works as an active element in the cell response [32]. Since the expression and activity of HAase are enhanced by proinflammatory cytokines such as IL-1β and TNF-α [33], the suppression of inflammation may be effective to inhibit the generation of HAoligo.

In conclusion, it is shown that HAoligo treatment results in increase in caspase-3 activity and decrease in Bcl-2 gene expression in cultured GFs, leading to the promotion of apoptosis. In addition, HAoligo generated under pathologic conditions may participate in the destruction of gingival tissue.


  1. Laurent, T.C. and Fraser, J.R. (1992) Hyaluronan. The FASEB Journal, 6, 2397-2404.
  2. Knudson, C.B. and Knudson, W. (1993). Hyaluronanbinding proteins in development, tissue homeostasis, and disease. The FASEB Journal, 7, 1233-1241.
  3. Yamalik, N., Caglayan, F., Kilinc, K., Kilinc, A. and Tumer, C. (2000) The importance of data presentation regarding gingival crevicular fluid myeloperoxidase and elastase-like activity in periodontal disease and health status. Journal of Periodontology, 71, 460-467. doi:10.1902/jop.2000.71.3.460
  4. Bartold, P.M. and Page, R.C. (1986) Hyaluronic acid synthesized by fibroblasts cultured from normal and chronically inflamed human gingivae. Collagen and Related Research, 6, 365. doi:10.1016/S0174-173X(86)80006-1
  5. McKee, C.M., Penno, M.B., Cowman, M., Burdick, M.D., Strieter, R.M., Bao, C. and Noble, P.W. (1996) Hyaluronan (HA) fragments induce chemokine gene expression in alveolar macrophages. The role of HA size and CD44. The Journal of Clinical Investigation, 98, 2403-2413. doi:10.1172/JCI119054
  6. Oertli, B., Beck-Schimmer, B., Fan, X. and Wuthrich, R.P. (1998) Mechanisms of hyaluronan-induced up-regulation of ICAM-1 and VCAM-1 expression by murine kidney tubular epithelial cells: Hyaluronan triggers cell adhesion molecule expression through a mechanism involving activation of nuclear factor-kB and activating protein-1. The Journal of Immunology, 161, 3431-3437.
  7. Nakatani, Y., Tanimoto, K., Tanaka, N., Tanne, Y., Kamiya, T., Kunimatsu, R., Tanaka, E. and Tanne, K. (2009) Effects of hyaluronan oligosaccharide on the expression of MMP-1 in periodontal ligament cells. Archives of Oral Biology, 54, 757-763. doi:10.1016/j.archoralbio.2009.05.005
  8. Gamonal, J., Bascones, A., Acevedo, A., Blanco, E. and Silva, A. (2001) Apoptosis in chronic adult periodontitis analyzed by in situ DNA breaks, electron microscopy, and immunohistochemistry. Journal of Periodontology, 72, 517-525. doi:10.1902/jop.2001.72.4.517
  9. Sorkin, B.C. and Niederman, R. (1998) Short chain carboxylic acids decrease human gingival keratinocyte proliferation and increase apoptosis and necrosis. Journal of Clinical Periodontology, 25, 311-315. doi:10.1111/j.1600-051X.1998.tb02446.x
  10. Cohen, J.J. (1991) Programmed cell death in the immune system. Advances in Immunology, 50, 55. doi:10.1016/S0065-2776(08)60822-6
  11. Kluck, R.M., Bossy-Wetzel, E., Green, D.R. and Newmeyer, D.D. (1997) The release of cytochrome c from mitochondria: A primary site for Bcl-2 regulation of apoptosis. Science, 275, 1132. doi:10.1126/science.275.5303.1132
  12. Susin, S.A., Lorenzo, H.K., Zamzami, N., Marzo, I., Snow, B.E., Brothers, G.M., Mangion, J., Jacotot, E., Costantini, P. and Loeffler, M. (1999) Molecular characterization of mitochondrial apoptosis-inducing factor. Nature, 397, 441-446. doi:10.1038/17135
  13. Martinou, J.C., Desagher, S. and Antonsson, B. (2000) Cytochrome c release from mitochondria: All or nothing. Nature Cell Biology, 2, 41-42. doi:10.1038/35004069
  14. Nunez, G., Benedict, M.A., Hu, Y. and Inohara, N. (1998) Caspases: The proteases of the apoptotic pathway. Oncogene, 17, 3237. doi:10.1038/sj.onc.1202581
  15. Tsujimoto, Y. and Shimizu, S. (2000) Bcl-2 family: Lifeor-death switch. FEBS Letters, 466, 6-10. doi:10.1016/S0014-5793(99)01761-5
  16. Watanabe, Y., Yamamoto, K., Kono, R. and Nunoda, D. (2006) Effects of intraarticular injection of hyaluronic acid in a rabbit osteoarthritis model. Journal of Tokyo Medical University, 64, 332.
  17. Ghatak, S., Misra, S. and Toole, B.P. (2002) Hyaluronan oligosaccharides inhibit anchorage-independent growth of tumor cells by suppressing the phosphoinositide 3- kinase/Akt cell survival pathway. The Journal of Biological Chemistry, 277, 38013-38020. doi:10.1074/jbc.M202404200
  18. Underhill, C.B. and Toole, B.P. (1979) Binding of hyaluronate to the surface of cultured cells. The Journal of Cell Biology, 82, 475-484. doi:10.1083/jcb.82.2.475
  19. Bartold, P.M., Boyd, R.R. and Page, R.C. (1986) Proteoglycans synthesized by gingival fibroblasts derived from human donors of different ages. Journal of Cellular Physiology, 126, 37-46. doi:10.1002/jcp.1041260106
  20. Breen, M., Weinstein, H.G., Johnson, R.L., Veis, A. and Marshall, R.T. (1970) Acidic glycosaminoglycans in human skin during fetal development and adult life. Biochimica et Biophysica Acta, 201, 54-60. doi:10.1016/0304-4165(70)90009-7
  21. Kawakami, H. and Terayama, H. (1981) Liver plasma membranes and proteoglycan prepared therefrom inhibit the growth of hepatoma cells in vitro. Biochimica et Biophysica Acta (BBA)-Biomembranes, 646, 161-168. doi:10.1016/0005-2736(81)90283-2
  22. Fritze, L.M., Reilly, C.F. and Rosenberg, R.D. (1985) An antiproliferative heparan sulfate species produced by postconfluent smooth muscle cells. The Journal of Cell Biology, 100, 1041-1049. doi:10.1083/jcb.100.4.1041
  23. Fujii, K., Fujii, Y., Hubscher, S. and Tanaka, Y. (2001) CD44 is the physiological trigger of Fas up-regulation on rheumatoid synovial cells. The Journal of Immunology, 167, 1198-1203.
  24. Brockhaus, F. and Brune, B. (1998) U937 apoptotic cell death by nitric oxide: Bcl-2 downregulation and caspase activation. Experimental Cell Research, 238, 33-41. doi:10.1006/excr.1997.3778
  25. Messmer, U.K., Reed, J.C. and Brune, B. (1996) Bcl-2 protects macrophages from nitric oxide-induced apoptosis. Journal of Biological Chemistry, 271, 20192. doi:10.1074/jbc.271.33.20192
  26. Knudson, C.B. and Knudson, W. (2004) Hyaluronan and CD44: Modulators of chondrocyte metabolism. Clinical Orthopaedics and Related Research, 427, 152. doi:10.1097/01.blo.0000143804.26638.82
  27. Lisignoli, G., Grassi, F., Zini, N., Toneguzzi, S., Piacentini, A., Guidolin, D., Bevilacqua, C. and Facchini, A. (2001) Anti-fas-induced apoptosis in chondrocytes reduced by hyaluronan: Evidence for CD44 and CD54 (intercellular adhesion molecule 1) involvement. Arthritis & Rheumatism, 44, 1800-1807. doi:10.1002/1529-0131(200108)44:8<1800::AID-ART317>3.0.CO;2-1
  28. Yamalik, N., Kilinc, K., Caglayan, F., Eratalay, K. and Caglayan, G. (1998) Molecular size distribution analysis of human gingival proteoglycans and glycosaminoglycans in specific periodontal diseases. Journal of Clinical Periodontology, 25, 145-152. doi:10.1111/j.1600-051X.1998.tb02420.x
  29. Moseley, R., Waddington, R., Evans, P., Halliwell, B. and Embery, G. (1995) The chemical modification of glycolsaminoglycan structure by oxygen-derived species in vitro. Biochimica et Biophysica Acta (BBA)-General Subjects, 1244, 245-252. doi:10.1016/0304-4165(95)00010-9
  30. Tanimoto, K., Ohno, S., Fujimoto, K., Honda, K., Ijuin, C., Tanaka, N., Doi, T., Nakahara, M. and Tanne, K. (2001) Proinflammatory cytokines regulate the gene expression of hyaluronic acid synthetase in cultured rabbit synovial membrane cells. Connective Tissue Research, 42, 187-195. doi:10.3109/03008200109005649
  31. Flannery, C.R., Little, C.B., Hughes, C.E. and Caterson, B. (1998) Expression and activity of articular cartilage hyaluronidases. Biochemical and Biophysical Research Communications, 251, 824-829. doi:10.1006/bbrc.1998.9561
  32. Ohno-Nakahara, M., Honda, K., Tanimoto, K., Tanaka, N., Doi, T., Suzuki, A., Yoneno, K., Nakatani, Y., Ueki, M. and Ohno, S. (2004) Induction of CD44 and MMP expression by hyaluronidase treatment of articular chondrocytes. Journal of Biochemistry, 135, 567-575. doi:10.1093/jb/mvh069
  33. Ohno, S., Ijuin, C., Doi, T., Yoneno, K. and Tanne, K. (2002) Expression and activity of hyaluronidase in human periodontal ligament fibroblasts. Journal of Periodontology, 73, 1331-1337. doi:10.1902/jop.2002.73.11.1331