Although he was a churchman, Master Richard of Gloucester had a long-term relationship with a resident of London called Katherine, the daughter of Geoffrey and Isabella of St Albans. This relationship produced at least two sons: John, born in the summer of 1317, and Nicholas, born in the autumn of 1319. In October 1329 a few months after Richard's death, the mayor and aldermen of London granted Katherine custody of her and Richard's two sons, then aged twelve and ten.  In his will, Richard left Katherine his house on Friday Street in London for the rest of her life, and he also mentioned their two sons, though he only called them Katherine's sons and not his. That they in fact were his children is, however, apparent, and this was obviously widely known at the time. In 1342, Nicholas, the younger son, called himself "Nicholas of Gloucester, son of Katherine of St Albans", using his father's last name, and in 1338 John was referred to as "son of the late Master Richard of Gloucester". In his will, Richard bequeathed "to John her [Katherine's] son a certain hall erected on a stage over the street [Friday Street], together with a shop", and "to Nicholas, son of the said Katherine, a tenement in the parish of S. Brigid for life, except a portion sufficient for the maintenance of a chaplain to celebrate for the good of his [Richard's] soul and the souls of others." Richard also left a "certain tenement" to John of Gloucester, whom he called his "kinsman" and who would appear to have been his brother or nephew.
Richard's inquisition post mortem of 1329 states that he held the manor of Woolwich "with remainder to John, parson of the church of Erdyngton, Adam son of Katherine de Sancto Albano [i.e. of St Albans], and Nicholas brother of the said Adam." The 1332 will of John of Gloucester, 'rector of Herdyngton', also survives, and talks of "the soul of Master Richard of Gloucester", though does not clarify the family relationship.  The 'Adam' mentioned here is a bit confusing, as no Adam, son of Katherine of St Albans, is mentioned in Richard's will; possibly this is a clerical error and meant Richard and Katherine's son John (who is not otherwise mentioned in the IPM), or possibly they had a third son together, or possibly Adam was Katherine's son from another relationship. By June 1342, Richard and Katherine's son Nicholas of Gloucester, born c. September/October 1319 and then twenty-two years old, was "lord of the manor of Southall Marreys", i.e. Woolwich.  His elder brother John was a bad lot, evidently: in August 1338 at a congregation of the mayor, aldermen and commonalty of London, it was decided that John, openly named as Master Richard's son, "and other incorrigibles should be committed to Newgate to prevent their doing mischief." 
I haven't been able to find much about Katherine of St Albans, though there's a reference in the Feet of Fines for London and Middlesex in the eighteenth year of Edward II's reign (July 1324 to July 1325) to "Master Richard de Gloucestr', John de Gloucestr', parson of the church of Herdington, and Katherine, daughter of Isabella de Sancto Albano."  Katherine appears on the Close Roll on 24 March 1337: "William de Pilardyngton, lord of Yeddyngg, lately granted by charter to Master Richard de Gloucestr', clerk, Sir John de Gloucestr', parson of Herdyngton church, and Katherine, daughter of Isabella de Sancto Albano, and to Richard her son and the legitimate heirs of his body" properties and fields in the town of Yeddyngg, i.e. Yeading, Middlesex. Although Katherine was still alive then, her son Richard was already dead without heirs of his body. 
Katherine of St Albans, therefore, had sons John and Nicholas, who were certainly also the sons of Master Richard of Gloucester, possibly a son Adam (unless he was a clerical error) who was named in Richard's IPM but not in his will, and another son, Richard, whom she presumably named after his father, Master Richard, or in his honour if Master Richard was not the father. Unfortunately I haven't been able to find any other references to Katherine or to her sons except the ones cited in this post. Her relationship with Master Richard evidently was a long and serious one, albeit illicit given his position, and he made sure that she and their sons were well provided for after his death. To my mind, whatever you might think about a parson's affair with a woman, this does him credit. I haven't found any references to Richard being taken to task over having a sexual relationship, perhaps surprisingly given that a good number of people in London seem to have known about it.
1) Calendar of Patent Rolls 1321-24, p. 352; CPR 1324-27, p. 1.
2) CPR 1321-24, p. 426; CPR 1324-27, p. 1; Calendar of Chancery Warrants 1244-1326, pp. 548-9; Pierre Chaplais, ed., The War of Saint-Sardos (1323-1325): Gascon Correspondence and Diplomatic Documents, pp. 5-6, 12, 177-8, 189, 191.
3) CPR 1272-81, p. 398; CPR 1327-30, p. 301.
4) Calendar of Inquisitions Post Mortem 1327-36, no. 223.
5) Calendar of the Select Plea and Memoranda Rolls of the City of London, vol. 1, 1323–64, p. 14.
6) CIPM 1327-36, no. 223; Calendar of Fine Rolls 1327-37, p. 119; Calendar of Wills Proved and Enrolled in the Court of Husting, London, part 1, 1258–1358, p. 342.
7) Calendar of Letter-Books of the City of London, Letter-Book E, p. 239.
8) Calendar of Wills Proved and Enrolled, p. 382.
9) Calendar of Letter-Books of the City of London, Letter-Book F, p. 75.
10) Select Plea and Memoranda Rolls, vol. 1, p. 168.
11) Calendar to the Feet of Fines for London and Middlesex, vol. 1, no. 333.
12) Calendar of Close Rolls 1337-39, p. 115.