The Drinfeld double of a finite quasi-Hopf algebra $H$ is a quasi-Hopf algebra $D(H)$ whose module category is the center of the monoidal category of $H$-modules; in this respect the double in the quasi-Hopf case behaves precisely like in the ordinary Hopf case, and this characterization was made the definition of $D(H)$ by Majid. However, since the dual of $H$ is not an associative algebra, it is not at all clear why $D(H)$ can be modelled on the vector space $H\otimes H^*$. In a sense this is the problem of exhibiting a basis for $D(H)$, which can more easily be defined in terms of generators and relations. We discuss the specific difficulties in the construction of $D(H)$ that arise in the quasi-Hopf case, and a new approach to overcoming them: Compared to the results of Hausser and Nill, we can avoid explicit calculations to a large extent, replacing them by rather conceptual considerations of suitable Hopf module categories.