A crack under a window often times a shrinkage crack, which are often vertical and appear a few months after constructing your foundation. As concrete dries, it shrinks and begins to crack, usually under your windows. This type of crack is settlement crack.
To deal with this type of crack, the right thing to do is to excavate the foundation till you get to a stable ground and underpin it with concrete to stop the settlement.
There are lots of advises on these type of issues out there to homeowners to open up the crack and fill it up with mortar, lime or resin grout. This is good but it must be done after you have identified the root cause of the problem and solved the problem before applying any fix on the crack.