cucumber

Comments and changes to this ticket

• 

  Joseph Wilk December 3rd, 2008 @ 09:40 PM

  • → Tag changed from “” to “featurerequest”
  • → Assigned user changed from “Aslak Hellesøy” to “Joseph Wilk”
• 

  Joseph Wilk December 5th, 2008 @ 01:08 PM

  • → State changed from “new” to “open”

  This is my current output when using: pending "Dude like stop it now!" in When /I press (.+)/

  
    Scenario: Add two numbers                      # examples/i18n/en/features/addition.feature:6
      Given I have entered 50 into the calculator  # examples/i18n/en/features/step_definitons/calculator_steps.rb:12
      And I have entered 70 into the calculator    # examples/i18n/en/features/step_definitons/calculator_steps.rb:12
      When I press add                             # examples/i18n/en/features/addition.feature:9
      Then the result should be 120 on the screen  # examples/i18n/en/features/step_definitons/calculator_steps.rb:21
      And the result class should be Fixnum        # examples/i18n/en/features/step_definitons/calculator_steps.rb:25
  
      |input_1|input_2|button|output|class |
      |20     |30     |add   |50    |Fixnum|
  
  TODO:
  When /I press (\w+)/ (Dude like stop it now!) # examples/i18n/en/features/step_definitons/calculator_steps.rb:16
  
  4 steps passed
  4 steps skipped
  2 steps pending
  

  WDYT?

  I've tried to keep it close to how Rspec's rendering/location for pending.

• 

  James B. Byrne December 5th, 2008 @ 03:47 PM

  • → Tag changed from “featurerequest” to “cucumber featurerequest”

  What is output for an unimplemented step?

• 

  Joseph Wilk December 5th, 2008 @ 05:45 PM

  Sorry that was not a great example. I've taken some of the ideas from a similar ticket I've been working on ( #52 ).

  Here is a better example of the output that I've refined. Unimplemented steps are shown in the bottom snippets and the pending tally shows the number of unimplemented steps.

  
    Scenario: Add two numbers                      # examples/i18n/en/features/addition.feature:6
      Given I have entered 50 into the calculator  # examples/i18n/en/features/step_definitons/calculator_steps.rb:12
      And I have entered 70 into the calculator    # examples/i18n/en/features/step_definitons/calculator_steps.rb:12
      When I press add                             # examples/i18n/en/features/addition.feature:9
      Then the result should be 120 on the screen  # examples/i18n/en/features/addition.feature:10
      And the result class should be Fixnum        # examples/i18n/en/features/step_definitons/calculator_steps.rb:21
  
      |input_1|input_2|button|output|class |
      |20     |30     |add   |50    |
  
  Todo:
  When /I press (\w+)/ (Dude like stop it now!) # examples/i18n/en/features/step_definitons/calculator_steps.rb:16
  
  4 steps passed
  2 steps skipped
  4 steps pending (2 with no step definition)
  
  You can use these snippets to implement pending steps which have no step definition:
  
  Then /^the result should be 120 on the screen$/ do
  end
  

  • I was trying to keep to number of step states to passed/skipped/pending to keep it simple.

  • Snippets are often used when you first start development. Pending steps raised through 'pending' are likely to hang around for longer. Hence It felt like it was a good idea to have them above the results. That's exactly what Rspec does.

  WDYT?

  I'm not sure 'Todo' is the right phrase. I've been playing around with 'Pending notes' as well.

• 

  James B. Byrne December 5th, 2008 @ 08:24 PM

  This will certainly serve. I question whether limiting the number of step categories is itself of value.

  I gather from #52 that one of the issues to address is what happens when step matchers are corrupted during a refactoring exercise. The desire being that, as missing matchers are different in nature to pending matchers, they are reported in that manner. It does not seem to me that overloading pending with the missing steps, which are in any case detailed separately in your example, is superior than simply having an additional one line of output that says that explicitly.

  
  TODO:
   Pending Steps here
  
  
  4 steps passed
  2 steps skipped
  2 steps pending
  2 steps undefined
  
  You can use these snippets to implement pending steps which have no step definition:
  
    Missing step definitions here
  

  versus

  
  
  TODO:
    Pending Steps here
  
  
  4 steps passed
  2 steps skipped
  4 steps pending (2 with no step definition)
  
  You can use these snippets to implement pending steps which have no step definition:
  
    Missing step definitions here
  

  I like the idea of putting the pending step definitions and their text above the summary report and the missing step definition text snippets below.

  I wonder if we should do something about the inverse case and also report orphaned step definitions, those without corresponding feature steps, or at least provide a separate option that does that on demand.

• 

  James B. Byrne December 5th, 2008 @ 09:35 PM

  How are missing step definitions colourized? As for pending? Would using orange instead of yellow for feature steps that lack a matching step definition be a worth while addition?

• 

  Joseph Wilk December 7th, 2008 @ 09:36 PM

  • → Milestone changed from “” to “0.1.13”
  • → Tag changed from “cucumber featurerequest” to “featurerequest”

  At the moment black. Unfortunately we cannot use orange. The only colours we support are:

  • black
  • red
  • green
  • yellow
  • blue
  • magenta
  • cyan
  • white
  • grey

  You can play with what I have done so far at my github repo: http://github.com/josephwilk/cuc...

• 

  James B. Byrne December 8th, 2008 @ 03:52 PM

  Well, Red == Fail And Green == Pass And Yellow == Pending

  So Black == Missing seems fine.

  What do you think about Grey? Grey seems to be the conventional colour used to indicate unimplemented or inactive functionality. In certain terminal colour schemes Grey might be a bit difficult to read, but then again, Black in a white on black session will be invisible.

  Considering the range of possibilities for user session colour schemes, might the Pending and Missing Step colours, or actually all report colours, be available user options in the cucumber.yml file?

  
  # cucumber.yml
  default: -r features --color --format-pretty --step_pending=Yellow --step_missing=Black --step_passing=Green --step_failing=Red
  autotest: -r features --color
  

  Selecting colours is moving away from the original request, so perhaps a new ticket is in order. But what do you think of the idea itself?

• 

  Joseph Wilk December 8th, 2008 @ 04:07 PM

  I don't think its been documented yet but you can actually already customize the colours by setting an env variable:

  ENV['CUCUMBER_COLORS']='passed=bold,green:passed_param=green:failed=red'

  These are the actions that can have their colour assigned.

  • passed
  • passed_param
  • failed
  • failed_param
  • skipped
  • skipped_param
  • pending
  • comment
• 

  James B. Byrne December 8th, 2008 @ 04:59 PM

  Well, this is certainly very useful to discover. What do you think of exposing this capability through the cucumber.yml file? Or, do you think that the ENV setting is a better approach?

• 

  James B. Byrne December 8th, 2008 @ 05:05 PM

  I tried your fork (0.1.13) and this is what I am seeing:

  
  # cucumber features -r features
  /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `gem_original_require': no such file to load -- cucumber/world (LoadError)
          from /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `require'
          from /usr/lib/ruby/gems/1.8/gems/cucumber-0.1.13/bin/../lib/cucumber.rb:18
          from /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `gem_original_require'
          from /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `require'
          from /usr/lib/ruby/gems/1.8/gems/cucumber-0.1.13/bin/../lib/cucumber/cli.rb:2
          from /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `gem_original_require'
          from /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `require'
          from /usr/lib/ruby/gems/1.8/gems/cucumber-0.1.13/bin/cucumber:5
          from /usr/bin/cucumber:19:in `load'
          from /usr/bin/cucumber:19
  
  

  I have made no changes to the features directory tree and that tree is processed by 0.1.12 without these errors.

  I built the 0.1.13 gem from a clone of your fork using # rake install_gem

• 

  Aslak Hellesøy December 8th, 2008 @ 05:08 PM

  Colours are developer-specific, not project specific and should not be committed to scm. Therefore it doesn't belong in cucumber.yml, but in ENV.

  On Mon, Dec 8, 2008 at 4:59 PM, Lighthouse no-reply@lighthouseapp.comwrote:

• 

  Joseph Wilk December 8th, 2008 @ 05:15 PM

  The problem is in the gemspec, I generally avoid committing against it.

  Running this will get you started:

  rake gemspec:refresh gem

  sudo gem install pkg/cucumber-0.1.13.gem

• 

  James B. Byrne December 8th, 2008 @ 06:53 PM

  Tried that. No luck. See the attached file for the process I followed and the results obtained. Perhaps I missed something.

  • cucumber_wilk_gemspec.txt 1.8 KB
• 

  Joseph Wilk December 8th, 2008 @ 07:08 PM

  Ok, I've pushed the gemspec which now contains the missing files (world/pending) to my repo. If you do a pull and build the gem it should work. Let me know if you have any problems.

• 

  James B. Byrne December 8th, 2008 @ 07:37 PM

  0.1.13 works for me now. Out of the box it shows unimplemented steps in yellow, not black as I gathered was the case from your previous remarks. Perhaps I misunderstood? V-0.1.13 also places the first Scenario immediately below the last line of the feature. I thought that earlier versions put in a blank line between the end of the feature statements and the first scenario in a feature file but, perhaps not.

• 

  James B. Byrne December 8th, 2008 @ 10:09 PM

  I looked at ansicolor.rb for the CUCUMBER_COLORS and then, not finding what I was expecting, I re-read this thread in its entirety. Unless I have missed it, there is no provision for a separate category of step reporting, namely:

  • missing

  Is there a technical limitation or other factor that requires/makes desirable reporting/highlighting missing steps as simply a special case of pending? I appreciate that the number of missing steps is now reported and that their snippets are collated separately. However, I gather from #52 that is it desired that the missing steps themselves somehow be distinguished.

  I gather from step_mother.rb that you are treating missing steps as pending and steps with a pending call in their block as a forced pending. I wonder if I am missing something in this. Would it not be more natural to simply consider and report undefined steps as "missing" and only treat defined steps with the pending call as "pending"?

  I understand that prior to this feature request missing steps were considered "pending", or rather step_matchers without blocks were, and that this was in fact the only way to have a pending step. And I can see the desire perhaps, of not disrupting this convention for those people used to this. I think though that distinguishing the two completely, and not simply considering one as a special case of the other, is the best way to proceed.

  What do you think?

• 

  Joseph Wilk December 23rd, 2008 @ 12:35 PM

  • → State changed from “open” to “resolved”

  Sorry for the delay in getting back to you.

  Within this specific ticket I really focused on introducing the pending method rather than the distinction of steps. There is no particular technical reason limiting us from treating pending and non implemented steps differently.

  I would like to close this issue as 'adding pending (or perhaps Pending) method' has been added. As for the discussion of separation and how that is presented I would like to address this is in a new issue ( #146 ).

  As you mention this comes down to peoples workflow.

  • Do people want to know the difference (#52 suggest sometimes yes)
  • How often do people need to know the difference?
  • How would people like this difference to be communicated?

  These are all questions I would like to open for further discussion with the community. This was my primarily reason for not implementing the distinction straight out.

• 

  James B. Byrne December 23rd, 2008 @ 02:56 PM

  I understand. I actually picked up the idea to distinguish the two from ticket #52. Having done so I could see its usefulness when refactoring features or definitions. Do you want me to open a ticket requesting a feature to allow users to distinguish between pending and unimplemented steps?
  

  Actually, it might be sufficient to simply have some sort of sweep function that just reports feature steps that have no step definition matchers. A regular run of cucumber retains the present behaviour but the user can select to report all missing step definitions as a separate task.

  What do you think?

• 

  Joseph Wilk December 23rd, 2008 @ 05:02 PM

  Looks like Aslak has already solved the problem for us in the upcoming development branch (ast).

  http://rspec.lighthouseapp.com/p...

  Which allows the developer to set specific colours for missing and pending steps.