Optimizing renewable energy usage in smart microgrids that contain photovoltaic production and battery storage is important due to the potential to reduce overall CO2 emissions and thus lead to more environmental friendly energy systems. However, due to the complex nature of energy management in smart grids and the volatile nature of energy production from PV systems, the problem is complex to solve. In this work we aim to optimize the energy in a microgrid comprising six houses using a digital twin based approach based on Deep Reinforcement Learning. We develop a Soft Actor-Critic (SAC) agent to address this intricate challenge, with the aim to simultaneously reduce emissions, maintain user comfort, while maximizing grid efficiency and resiliency to cope with spurious grid outages. We propose and evaluate different reward functions that guide the agent in finding its optimal policy. Furthermore, we discuss the implications of our results and outline potential future steps, envisioning ongoing refinement and advancements in our pursuit of optimal solutions for the complex interplay of severaal objectives in microgrid management.