The Problem: Why Space-Separated Parsing Fails in Boost.Spirit Qi

When parsing structured data with C++ and Boost.Spirit Qi, you often use a skipper (like ascii::space_type) to automatically ignore whitespace between tokens. However, this can lead to unexpected behavior when you try to parse a collection of items explicitly separated by spaces using the list operator (%).

Consider the following parser rule:

start = int_ >> (employeeRule % ' ');

If your input string is space-separated (e.g., "10 employee {...} employee {...}"), this parser will fail. Why? Because the skipper greedily consumes the space character before the list operator (%) can match it as a delimiter. Since the space is already eaten, the parser looks for a literal space, finds the next character of the struct (like e from employee), and fails to match.

The Solution: Let the Skipper Do the Work

Since your grammar already uses ascii::space_type as a skipper, you do not need to explicitly match spaces as delimiters. The skipper automatically handles any whitespace (spaces, tabs, newlines) between your elements.

Instead of using the list operator (%) with a space delimiter, you should use the Kleene star (*) or the plus operator (+) to match one or more occurrences of your struct sequentially. Boost.Spirit will automatically skip the spaces between them.

Corrected Code Example

Modify your start rule to use +employeeRule (which means "one or more") instead of the list operator:

#include <boost/spirit/include/qi.hpp>
#include <boost/fusion/include/adapt_struct.hpp>
#include <vector>
#include <string>
#include <iostream>

namespace qi = boost::spirit::qi;
namespace ascii = boost::spirit::ascii;

struct employee
{
    int age;
    std::string surname;
    std::string forename;
    double salary;
};

struct employees
{
    int dummy;
    std::vector<employee> collection;
};

BOOST_FUSION_ADAPT_STRUCT(
    employee,
    (int, age)
    (std::string, surname)
    (std::string, forename)
    (double, salary)
)

BOOST_FUSION_ADAPT_STRUCT(
    employees,
    (int, dummy)
    (std::vector<employee>, collection)
)

template <typename Iterator>
struct employee_parser : qi::grammar<Iterator, employees(), ascii::space_type>
{
    employee_parser() : employee_parser::base_type(start)
    {
        quoted_string %= qi::lexeme['"' >> +(qi::char_ - '"') >> '"'];

        employeeRule %=
            qi::lit("employee")
            >> '{'
            >> qi::int_ >> ','
            >> quoted_string >> ','
            >> quoted_string >> ','
            >> qi::double_
            >> '}'
            ;

        // SOLUTION: Use +employeeRule instead of (employeeRule % ' ')
        start = qi::int_ >> +employeeRule;
    }

    qi::rule<Iterator, std::string(), ascii::space_type> quoted_string;
    qi::rule<Iterator, employees(), ascii::space_type> start;
    qi::rule<Iterator, employee(), ascii::space_type> employeeRule;
};

int main()
{
    std::string str = "10 employee {10, \"Smith\", \"John\", 100000.0 } employee {12, \"Smith1\", \"John12\", 800000.0 }";
    
    typedef std::string::const_iterator iterator_type;
    typedef employee_parser<iterator_type> employee_parser;

    employee_parser parser;
    employees data;

    std::string::const_iterator iter = str.begin();
    std::string::const_iterator end = str.end();
    
    bool r = qi::phrase_parse(iter, end, parser, ascii::space, data);

    if (r && iter == end)
    {
        std::cout << "Parsing succeeded!\n";
        std::cout << "Parsed " << data.collection.size() << " employees.\n";
    }
    else
    {
        std::cout << "Parsing failed at: " << std::string(iter, end) << "\n";
    }

    return 0;
}

Why This Works

  • Skipper Integration: Because phrase_parse is called with ascii::space, Boost.Spirit automatically skips any whitespace before trying to match the next employeeRule instance inside the + loop.
  • Simplified Rules: Removing the explicit delimiter makes the grammar simpler, cleaner, and less prone to edge-case bugs like trailing spaces.