The construction industry is considered an activity responsible for both significant CO2 emissions and material flows. Providing affordable, decent, and sustainable habitats is, however, a fundamental human need. Therefore, the chapter aims to analyse the environmental impacts of construction and energy upgrades of detached family dwellings typical in Hungary. For the analysis of environmental impact, the concept of ecological footprint (EF) was used. The main advantages of this method are that (1) results can be compared to the upper limits of sustainability, (2) figures are rather easy to understand for each stakeholder, and (3) the concept may take into consideration all impacts of human activities. The chapter, however, analyses only the EF of material use of constructions. The sample consists of data on five detached family houses, representing the housing in Hungary: (1) a new two-storey family house with a gable roof, (2) a new family house made of timber, and (3) an energy upgrade of a typical family dwelling built between 1950 and 1990, called “Hungarian cube”. The results show that the construction of new detached family dwellings has a significantly higher ecological footprint than renovating an existing one. Using non-conventional construction materials (e.g. timber), however, can decrease the ecological footprint of construction, but useful lifetime should be considered more deeply in this case. Volatile energy prices have highlighted the importance of the energy efficiency of the housing stock. Newly built dwellings have outstanding insulation and, therefore, have nearly zero emission during operation, but they are not the means of affordable housing due to their construction costs, and the construction itself has a significant environmental impact which is not outweighed by the energy savings during the useful lifetime. Therefore, retrofitting and even extension of dwellings can better contribute to the Sustainable Development Goals (SDGs).