How to set up Claude and structure URLs for the directory - Learning in Public day 4
In this post, I'll walk you through the process of structuring the URLs and website architecture for an online travel directory. You'll learn how to organize your content in a way that's SEO-friendly, logical for users, and easy to expand as your site grows.
When you're building a directory website or any kind of content-heavy site, planning the right URL structure and website architecture from the start is crucial. This lays the foundation for how your content will be organized, how users will navigate your site, and how search engines will crawl and index your pages.
As part of my 60-day "Building in Public" challenge, I'm creating a travel directory website for Bali. Today, we're going to map out the ideal URL structure and website architecture for this project. Let's dive in!
The Challenge
Creating an intuitive, scalable structure for a directory site with tons of content is no easy feat. You need to consider things like:
Search intent: How are users searching for different types of content?
Content categories: What are the main categories and subcategories of content?
URL hierarchy: How should URLs be nested to reflect the category structure?
Crawlability: Will search engines be able to easily crawl and index all pages?
Future growth: How can the structure accommodate new content as the site expands?
Getting this right from the start saves major headaches down the road.
Our Approach
To tackle this challenge, I'm using an AI assistant called Claude from Anthropic. I'll provide Claude with all the keyword research data, explain the project goals, and have it consult me on the ideal URL and site architecture.
Claude will analyze the data, consider technical SEO best practices, and map out a logical structure for content organization and URL paths. We'll get a detailed breakdown, visual diagrams, and even sitemaps and robots.txt files.
Let's get started!
Step 1: Set Up the Project
First, I created a new project in Claude and uploaded all the keyword research data from the past few days. This includes keywords and search volumes for things like hotels, villas, restaurants, activities, and more in Bali.
Next, I explained the overall goal of building a Bali travel directory website with categories for places to stay, dining, wellness, beach clubs, activities, and area guides.
Step 2: Define Project Instructions
To ensure Claude understands exactly what kind of output I need, I provided clear instructions:
"I want you to act as an SEO consultant and project manager. Consult me on the ideal content architecture, URL structure, sitemaps, robots.txt, and more for building this directory. Always provide data-driven recommendations and clear, actionable advice."
This sets the expectations for Claude to strategically map out the entire technical foundation of the website.
Step 3: Analyze Data and Provide Recommendations
With the context and instructions set, Claude dug into the keyword data to understand how users are searching for different types of content. It then provided an initial recommendation for the main content categories and subcategory structures:
Stays: Hotels, Villas, Resorts, Glamping
Dining: Cuisine (Italian, Japanese, etc.), Meal Type (Breakfast, Dinner, etc.), Location
Wellness: Spas, Massage, Yoga, Gyms
Beach Clubs: Location
Activities: Tours, Cultural, Adventure, Location
Area Guides
We went back and forth a bit, refining the categories and discussing the ideal URL structure. The key considerations were keeping URLs short but descriptive, ability to filter listings, and crawlability for search engines.
Step 4: Map Out URL and Site Architecture
After aligning on the content architecture, Claude provided a comprehensive breakdown of how URLs should be structured:
Homepage
/stays
/stays/hotels
/stays/hotels/location
/stays/villas
/stays/resorts
/stays/glamping
/dining
/dining/cuisine/italian
/dining/cuisine/japanese
/dining/mealtype/breakfast
/dining/mealtype/dinner
/dining/location/ubud
/wellness
/wellness/spas
/wellness/massage
/wellness/yoga
/wellness/gyms
/beach-clubs
/beach-clubs/location
/activities
/activities/tours
/activities/cultural
/activities/adventure
/activities/location
/guidesThis structure keeps URLs clean and logical, while allowing for filtering by location, cuisine, price range, and other parameters.
Claude also provided sitemap files for organizing URLs in a search engine-friendly way, a robots.txt file for controlling crawlers, and diagrams visualizing the site architecture.
Challenges We Faced
One challenge was determining the right approach for location-specific pages. For example, should we create dedicated pages for "Italian restaurants in Ubud" if that's a high-traffic search?
Claude recommended starting with broader category pages that can be filtered, then monitoring analytics to see if dedicated location pages are needed. If so, we can use an admin feature to easily publish those high-value pages.
How We Overcame Them
Claude's flexible, data-driven approach allowed us to create a scalable architecture that can adapt as usage patterns emerge. By tracking search data and engagement metrics, we can identify opportunities to add dedicated content pages over time.
Results So Far
By the end of today's session, we had a clear, comprehensive plan for structuring the website and URLs in an SEO-friendly way. This covers all the main sections like stays, dining, activities, and more.
With Claude's help, I now have URL mapping diagrams, sitemap files, robots.txt instructions, and database planning to hand off to the development team.
Key Takeaways
Start with thorough keyword research to understand how users are searching for your content
Plan your URL structure to match content architecture, keeping URLs short but descriptive
Use tools like AI assistants to analyze data and get expert recommendations
Create a flexible architecture that can expand over time as content and traffic grows
Monitor analytics to identify opportunities for dedicated content pages based on demand
What's Next
Now that we have the technical foundations in place, we'll move into planning the structure and content for individual page types. We'll analyze competitor pages, review our keyword research, and map out templates for listings, location pages, and more.
Stay tuned as we take this project to the next level! Let me know if you have any questions in the comments.
Want to Learn More?
Subscribe to my YouTube channel for weekly updates on the "Building in Public" challenge! You can also follow me on Twitter for more tips on entrepreneurship, marketing, and building businesses.