It may not be directly from your address book: if you know enough people who have linked-in accounts, are linked to you, and have fallen for one of the access routes, then a lot can no doubt be inferred about you from them.