VoteNet detects 3D objects directly from point cloud data by combining deep point set networks with Hough voting to overcome the challenge that object centroids can be far from surface points in sparse 3D data. The model achieves state-of-the-art results on ScanNet and SUN RGB-D datasets using purely geometric information without relying on color images.