I suggest that you cut a hardwood block at the correct angle for the available hardware, or have a bracket with the correct angle made at a welding shop.
Another alternative is to use a closet rod that hangs from brackets mounted to the rear wall of the closet, or brackets that mount to the underside of a shelf in the closet. (This hardware is available from closet organizer manufacturers.)
I don't know of any hardware that mounts on a side wall that is not parallel to the other side wall.