Returning from a cattle drive up the Salt Grass and Chisholm Trails to Abilene, where he had sold three thousand head of cattle, John T. Sargent, received a message on September 14, 1875, that a hurricane was ravishing the Texas Gulf Coast. Riding day and night, Sargent reached home to find his home destroyed and could not locate his wife or children. He later found his children in a tree and brought them to safety. Two days later, he located the bodies of his wife, Sarah Ann Hill Sargent (1837 - 1875) and his father, George. Both had drowned on September 17, 1875, during the storm, and were buried in the Sargent cemetery in Sargent, Texas. Broken-hearted, he made the decision to move to the town of Matagorda, Texas with his daughters, Fannie P., Sarah Ann, Amelia, Mary Elizabeth, Josephine and his only son, George Thomas Sargent. Sargent purchased the Samuel Rhodes Fisher (1794 – 1839) home in Matagorda. Fisher was an original signer of the Texas Declaration of Independence.

John T. Sargent assisted in the design of the new home planned for the Sargent family. The house was to be built in honor of his daughter, the new bride, Catherine Sargent Rugeley. For the longest period of time, the Sargent – Rugeley - Herreth home was known as "Miss Kate Rugeley’s home." John T. Sargent and his wife Jane would eventually reside in the home with their daughter Catherine and her husband James Rugeley.

James Walcott Rugeley had never liked the Victorian style of the home, and as the head of the household, made the decision to begin rebuilding the exterior of the home in 1913 to reflect the Prairie architectural style developed around the turn of the century by Frank Lloyd Wright. The remodeled home was completed in 1915. The style is tempered with classical Greek Revival elements. The redesign included a ten foot by fifty foot veranda with ten supporting columns. The uniqueness of the home is further emphasized through the use of extensive beveled glass; stained glass windows; customized copper door hardware; and beautiful brass and crystal chandeliers. Elegant proportioning and quality detail make this well-designed structure one of the finest residences in Bay City.

The exterior decking and interior floors are one and one-fourth inch heart of pine planks. The structure is fifty feet by fifty feet and rises forty feet from its brick foundation. The twelve by twelve inch foundation beams are made of cypress wood and the diagonal pine sheathing is covered with cypress shiplap. The lumber for the home was shipped to Bay City from Calcasieu Parish, Louisiana.

The gutters and downspouts are made of solid copper. The molding around the third story roofline, matches the patterning of the gutters. The window screens are hung on the inside and they slide up and down on runners and springs, instead of in and out on hinges. They bear the patent date of September 6, 1898. Each window has a functioning cypress shutter made to fit the window for protection against Gulf storms. Two cement hitching posts, with steal rings, flank the front walkway.

The house consists of approximately twenty-six rooms, counting bathrooms, pantries, storerooms and the great hall on the second story. There are seven exterior doors and sixty-four windows. You enter into the large living room through a massive door surrounded by beveled plate glass windows on each side. This room is twenty-four feet by thirty-four feet. A stately five foot by seven foot wide fireplace is positioned on the south wall. On the left of the fireplace is a beautiful, leaded, stained-glass window. Adjoining rooms may be closed off with massive cypress sliding doors. The ground floor also includes a study, dining room, kitchen, pantry, servery, wash room, bathroom, parlor and study. The ceilings on this floor are nine and a half feet tall.

Up the winding, paneled staircase is a great hallway on the second story with nine and a half foot ceilings. The twelve foot by forty foot gallery gives access to library, bedrooms, baths, back staircase and the staircase leading to the third floor. The second story doorways are nine feet tall, including transom windows and are opened using glass doorknobs. The third floor has nine and a half foot ceilings, with rooms arranged in a cross pattern, providing perfect cross ventilation as well as a magnificent view in all directions.

John T. Sargent and his wife Jane were active in the communities of Matagorda and later Bay City. Mr. Sargent was a charter member of the Bay City Masonic Lodge, No. 865, which was chartered on December 4, 1902. He was known throughout Matagorda County as a fine rancher and cattleman. His decedents still operate a cattle ranch near the community of Sargent which bears his name.

The Sargent - Rugeley families owned the home from 1905 to 1969. Two years after Catherine’s death, the home was sold to the current owners, Albert Casper Herreth, Jr. and his wife, Georgia Irene Rice Herreth on May 5, 1969.

Senate Bill 387, passed by the 2001 Texas Legislature, allows public school districts to issue a high school diploma to a person who left high school before graduation to serve in World War II. Herreth received his belated high school diploma, wearing a cap and gown, to a standing ovation, from Tidehaven I. S. D. as he fully participated in the graduation ceremony with the other class members of the Graduating Class of 2002.

Georgia Irene Rice was born on March 14, 1931 in Sweetwater, Texas to Ira Elton "Jack" Rice (1907 – 1992) and Mary Rena Ramzel Rice (1913 – 1996). Jack and Rena were married on February 23, 1929, at the home of her parents in Sweetwater, Texas. The family settled in Bay City, Texas in 1937.

Jack Rice’s family had made their home in Texas since the 1860s. Jack Rice was born on October 3, 1907 in Comanche County, Texas. Rena Ramzel was born in Sweetwater, Texas on June 30, 1913. Her family had made their home in Texas since the 1830s. The couple was very active in their community. They were members of the Knights of Columbus, Bay City Chamber of Commerce, "100" Club, Matagorda County Cancer Society, Matagorda County Federated Women’s Club and Woodmen of the World. Both served as president of the Woodmen of the World. Rena was a member of the Daughters of the Republic of Texas and The United Daughters of 1812. Jack was a charter member of the Cradle of Texas Chapter of the National Society and the Texas Society of the Sons of the American Revolution. Jack was active in the Girls’ Softball Association, and their field, "The Jack Rice Field," was named in his honor.

Georgia Rice Herreth worked as the church secretary, a clerk for the Selective Service and an agent for Statewide Modern Homes. In 1970 she received her certification as an instructor from the Ben Shaw School of Modeling in Houston. She operated her own studio from 1973 – 1976. Mrs. Herreth was the first woman elected to the City Council of Bay City, serving two terms, 1979 – 1983. She was awarded the Bay City Chamber of Commerce "Outstanding Woman of the Year" in 1990.

William Stiles, Sr. was born in Mecklenburg County, Virginia on March 26, 1769. He was married in Elbert County, Georgia on May 26, 1792 to Esther Hetty Vinson (1769 – 1840). She was born Feb 4, 1769, to Jesse Vinson (1717 - 1801) and Esther Dawson Vinson (1721 – 1820). William Stiles, Sr. and his family, crossed the Arbuckle Mountains and settled in "Old Miller" County, Arkansas. The family made their home at the mouth of Mulberry Creek on the Arkansas River during 1816-18. The famous botanist, Thomas Nuttall (1786 – 1859) wrote that he stayed in the home of William Stiles during that period. During the winter of 1818 - 1819 he moved near present day Fort Towson, moving next to Red River County, Texas. In 1821 Stiles and his family joined Stephen F. Austin (1793 – 1836) and Austin’s "Old Three Hundred" families who were going south. Stiles received a labor of land in Fort Bend County and died there June 2, 1836. He is buried in the family cemetery located in the George Ranch Historical Park which includes some of his original land. The Will of William Stiles, Sr. was the first filed in Fort Bend County. He made his living as a stockman. Hetty and William Stiles, Sr. were the parents of six children; Richard, John, Sr., William Jr., Nancy, Elizabeth and Hetty.

Son John Stiles, Sr. stayed in Red River County. On his way to the Alamo, Davey Crockett (1786 – 1836), spent his first night in Texas in the home of his old Tennessee friend, John Stiles. Stiles, Isiah D. Lawson (1806 – 1847), and another man were named by Capt. William Becknall (1787 – 1856) and Lt. James Clark (1799 – 1838) to have personal charge of Antonio López de Santa Anna (1794 – 1876) just hours after his capture following the Battle of San Jacinto. Gen. Sam Houston (1793 – 1863) knew John Stiles quite well and was quoted as saying, "The very man! I know that if anyone would try to take the life of Santa Anna, it would be over the dead body of John Stiles."