The present invention relates to a semiconductor device including a high speed element having a super-strained lattice structure and more particularly to a high speed transistor element having a super-strained crystal lattice. Various high speed elements are currently used in semiconductor integrated circuit devices and the like. Among them are a transistor having a super-strained crystal lattice and a transistor having a strained crystal lattice. For example, Japanese Patent Laid-Open No. 2004-317915 discloses a transistor having a super-strained crystal lattice in which a crystal having a compression strain or a tension strain is formed using crystals having different lattice constants, so as to create a super-strained structure. Here, the crystal lattice constant of a crystal depends on the size of the crystal and there is no crystal having an exact size. Therefore, the lattice constant will be denoted herein as an approximate value. The above super-strained transistor increases the electron and hole mobility to attain high speed operation and can greatly improve the operating frequency and the ON-OFF ratio of a transistor element as compared to a transistor in which crystals having an ordinary lattice structure are formed. The super-strained transistor can provide an improved high frequency operation, but does not necessarily ensure a high reliability. There is also a problem that the super-strained transistor may be broken by a temperature cycle test. This is because stress that is caused by lattice mismatching at the interface between, for example, SiO2 and Si crystals has a temperature dependency and becomes a cause of crystal defects (hereinafter referred to as a “crystal defect”) when the temperature is changed or the operation is repeatedly performed. It should be noted that, even in transistors having a super-strained structure, there is a technique of providing a high reliability by adding a SiGe compound to Si. Japanese Patent Laid-Open No. 2004-317915 discloses a transistor having a super-strained structure in which a single crystal Si layer having a predetermined lattice constant and a mixed crystal having a Si1-xGex compound (x>0) are stacked. In this case, dislocation is caused by the crystal mismatch between the Si layer and the Si1-xGex layer and can be relieved. As a result, the formation of the crystal defects in the transistor can be suppressed. As described above, it is important to relieve the crystal defects in order to provide a high reliability in a transistor having a super-strained structure. However, a transistor having a super-strained structure as described above still has room for improvement in this regard. For example, when the gate length is reduced, the stress becomes more local and more severe. It is thought that the amount of strain becomes smaller with this, but it is difficult to provide a highly reliable super-strained structure even if this is the case. When a high-frequency high-speed operation can be realized, a high-speed memory circuit, a high-speed logic circuit, a radio-frequency amplifier circuit, a wireless signal circuit and the like can be realized by using the elements. For this purpose, it is important that all elements in the circuit operate at high speeds, and therefore there is a demand for a reliable super-strained transistor having a higher operation speed than those of currently available transistors.