The defect of a cubic threefold with isolated singularities is a global invariant that measures the failure of Q-factoriality. From the work of Cheltsov, a cubic threefold with only nodal singularities is Q factorial if and only if there are at most 5 nodes. We investigate the defect of cubic threefolds with worse than nodal isolated singularities, and provide a geometric method to compute this global invariant. One can then compute the Mixed Hodge structure on the middle cohomology of the cubic threefold, in terms of the defect (a global invariant) and local invariants (Du Bois and Link invariants) determined by the singularity types. We then relate the defect to geometric properties of the cubic threefold, showing it is positive if and only if the cubic contains a plane or a rational normal cubic scroll. The focus of this work is to provide more insight into the existence of reducible fibers for compactified intermediate jacobian fibrations associated to a smooth (not necessarily general) cubic fourfold. This is joint work with Sasha Viktorova.