The Parramatta Lunatic Asylum, took over the buildings of the Female Factory in 1848. As the history of the Parramatta Lunatic Asylum states,
... by 1846 the Tarban Creek Asylum near Gladesville was overcrowded compelling the Colonial Government to look for a solution. In 1847 Governor Sir Charles Fitzroy agreed that "lunatic" and invalid convicts be placed in one institution.
1943 Aerial View of the Site showing the Male Wards
The layout of the buildings conformed with contemporary designs for hospitals, with main services located in the spine and dormitory wards and rooms connected to the centre block. The Central Male Block complex has largely retained this courtyard design. The central sections were demolished and rebuilt close to the original design in 1934-35. The Front Verandah, Dining and Day Rooms were preserved.  Day and dining rooms tended to have unbarred windows. These and other windows in ward blocks were altered in the late 1800s, and tall thin windows were installed, so that it was impossible for patients to climb out. Also of significance is the former Canteen, now Harriett Ward, which began construction in 1956. This single brick building has a roof clad in terracotta tiles and timber weatherboards. In 1955 the Royal Australian Historical Society donated some bricks that had been preserved from the Sydney Exhibition Building which had been destroyed by fire in 1882, and as a result the Canteen was built with recycled bricks. The After-Care Association - which ran the canteen from its opening in 1957 - relinquished control of the building in 1996. It is assumed that this is when the building was renovated and the canteen was moved and became the Phoenix Café.  The precinct also has a Gardner’s Cottage with a bush house and nurseries. This building which was formerly a Gardner’s Cottage, has painted brick walls and a corrugated iron roof. It was constructed in 1899, with additions made in 1910 by the Government Architect Walter Liberty Vernon.
The additions made were another drawing room to the west of the cottage, and an additional room added to the east. Walter Vernon who also designed the Art Gallery of New South Wales, and was responsible for additions to the Customs House building in Sydney, is known for using the most advanced materials and construction methods available at that time. Gardener’s Cottage was originally a timber framed and weatherboard structure with 2 brick chimneys. A detached wash room was located to the north west of the building.  The verandah and roof have now been extended and enclosed, and the chimneys and detached wash room were removed at some stage. Stained glass windows to the rear of the building are still intact. The former female block buildings includes the 'Kitchen Block' (now a nurse education area) built between 1881-1883. This building has painted brick walls, and a painted corrugated iron roof. There is also a free standing timber shelter with a corrugated roof behind the building. The southern section of the building previously contained a bakehouse, and was adjacent to a store room. There is a painted chimney in the southern section of the building. The building was converted into Occupation Work Rooms in 1928, and had further modifications in 1943. Uses of the building since the Second World War include being used as wards and sewing rooms. 
The Laundry Block and Boiler House was designed by the Colonial Architect James Barnet, and was completed in 1883. The Laundry and Boiler House were located in the Female Precinct, as laundry in the 1800s was considered solely women’s work. The laundry was often located near the boiling room and had four main components: "receiving and distribution rooms; wash house; ironing rooms and drying racks and closets"  The widespread idea at the time was that germs spread through the air, so the laundry was commonly kept at a distance from other buildings. The Laundry at the Male and Female Asylum was purposely built in the north of the Female Precinct.The Laundry Block was modified in 1901, and remains largely intact. The Female Precinct previously had a Timber Female Asylum. The former Female Factory buildings were considered inappropriate for the needs of the female asylum patients, and so were demolished. These were replaced with weatherboard asylums that were constructed in the 1870's and 1880's. This temporary female asylum conformed to the contemporary ideas for the design of hospitals and barracks with services located in the spine and dormitory wards and rooms connected to the centre block. The Female Timber Asylum was demolished during the 1970’s.
Caroline Finlay, Regional Studies Facilitator, Parramatta City Council, Heritage Centre, 2015
Footnotes  Higginbotham, E 2010, p.199  Higginbotham, E 2010, p.214  Higginbotham, E 2010, p.226  Higginbotham, E 2010, p.193  Department of Public Works and Services 1998, p.158 References Department of Public Works and Services, North Parramatta Government Sites, 1998. Edward Higgenbotham & Associates Pty Ltd, Geoffrey Britton & Terry Kass, Conservation Management Plan and Archaeological Management Plan, Cumberland Hospital East Campus and Wisteria Gardens, 2010.