Would this be a valid use of inheritance?

  Suppose you want to make a smart pointer which is allocator-conscious,
so that it can be told which allocator was used to allocate the object,
so that the smart pointer will know to use the same allocator to
deallocate it. The traditional "kosher" way of doing that would be
something along the lines of:

//---------------------------------------------------------------------
template<typename Data_t, typename Alloc_t = std::allocator<Data_t> >
class SmartPtr
{
    Data_t* data;
    Alloc_t alloc;
    // And whatever is needed for reference counting, if anything

    void decRefCount()
    {
        // Decrement the reference count, and then:
        if(<refcount is 0>)
        {
            alloc.destroy(data);
            alloc.deallocate(data, 1);
        }
    }

 public:
    SmartPtr(Data_t* d, const Alloc_t& a):
        data(d), alloc(a)
    {
        // init refcount, or whatever
    }

    // Plus the other necessary functions.
};
//---------------------------------------------------------------------

  The problem with this is that even if the specified allocator is an
empty class, it will take 4 bytes (I assume 8 bytes in a 64-bit system)
of space as a member variable. At least gcc seems unable to optimize
that space away, even though it's not used for anything. This needlessly
increases the size of the smart pointer, even though the allocator
doesn't need any space.

  So I was thinking, would this be a valid way of circumventing the problem?

//---------------------------------------------------------------------
template<typename Data_t, typename Alloc_t = std::allocator<Data_t> >
class SmartPtr: private Alloc_t
{
    Data_t* data;
    // And whatever is needed for reference counting, if anything

    void decRefCount()
    {
        // Decrement the reference count, and then:
        if(<refcount is 0>)
        {
            destroy(data);
            deallocate(data, 1);
        }
    }

 public:
    SmartPtr(Data_t* d, const Alloc_t& a):
        Alloc_t(a), data(d)
    {
        // init refcount, or whatever
    }

    // Plus the other necessary functions.
};
//---------------------------------------------------------------------

  I know this is badly misusing inheritance for an ugly hack, but now an
empty allocator class will not increase the size of the smart pointer,
but the smart pointer can still be told which allocator to use to
destroy the object.

  Can you think of any reason why this would be a very bad idea?