Your and Kaczynski's confusion about what the "left" constitutes might be helped by consideration of the "political compass". It plots your political position based on a questionnaire which assesses your location on two axes - economic (left/right) and social (libertarian/authoritarian). The very fact that libertarianism/authoritarianism can be assessed orthogonally to left/right should clear up most of your misconceptions.
For the record, I rate as centrist/libertarian, a position designated the "extreme left" along with most of the rest of the area on the chart by American interlocutors. I can assure you that plenty of people come down on the left/libertarian quadrant.