Gene regulatory networks in living cells are comprised of recurring network motifs, which perform key functions to control cellular responses. Mathematical models of network motifs are already developed and experimentally validated. We analyze the stability of previously validated mathematical models of network motifs against the fluctuations in their associated biochemical reaction rates and find that all of these elementary functional modules are stable against any perturbation in their rates of reaction. In gene regulatory networks the motifs are connected to each other in a directed acyclic manner. Any large assembly of stable and robust motifs connected with each other in a directed acyclic manner exhibits stable and robust dynamics. All the motifs except those having feedback loops in their structure preserve certain dynamic properties when embedded within large operational networks. Our study also suggests that evolutionary mechanisms selected stable and robust functional modules with which to build regulatory networks in order to ensure stability and reliability on a larger scale instead of finding the simplest canonical representation with which to serve the same purpose.