Heat a comal or skillet over medium heat and toast the guajillo and ancho chiles for about 1 minute, flipping them over a few times, until they become more pliable, lightly toasted, fragrant and their inner skin turns opaque. Transfer to a medium saucepan. Add the tomato, cover with water and bring to a simmer over medium heat. Simmer for 10 to 12 minutes, or until the tomato is very soft and the chiles are fully hydrated, plumped up and soft.
Place the chiles, tomato and 1/2 cup of the chile simmering water in a blender jar. Add the oregano, whole cloves, cinnamon, cumin and vinegar, and puree until smooth.
Heat the oil over medium high heat in a large, deep skillet. Add the onion and cook, stirring occasionally, until tender and the edges begin to brown, about 3 to 4 minutes. Add the garlic and cook, stirring, until fragrant, about 1 minute. Add the meat, season with salt and pepper, and cook, stirring occasionally, until lightly browned, about 8 to 10 minutes.
Reduce heat to medium, pour the chile puree over the meat, and stir in the broth. Add the raisins, almonds, olives and brown sugar, stir together, reduce the heat to medium low, cover and cook for 12 to 15 minutes, stirring occasionally. The mixture should cook down and have the consistency of chile con carne.