Skip to content

Conversation

pleroy
Copy link
Contributor

@pleroy pleroy commented Jun 7, 2014

Apologies for the size of this pull request but it seems hard to split it in a meaningful way.

In porting the benchmark library I have tried to follow the model used by glog, and I have lifted some code from there. Here is an overview of the changes:

  1. Add a subdirectory msvc/ containing the MS project and solution files. There is one project for the library (google-benchmark) and one project for the tests (google-benchmark-test). This stuff is just humongous blobs of automatically-generated XML, not much to review.
  2. Change .gitignore and .gitattributes to properly handle the MSVC artifacts.
  3. Add a src/port.h file (and the corresponding src/port.cc) to factor out the differences between Windows and the various Unix-ish OSes. I lifted this file from glog and did the necessary adjustments. In particular, it defines a bunch of HAVE_ symbols that act as a poor man's config.h. src/port.h must be included early in every source file that has OS dependencies, as among other things it defines the all-important symbols OS_WINDOWS and COMPILER_MSVC.
  4. Change a bunch of includes to make them depend on the HAVE_ symbols.
  5. Implement MyCPUUsage for Windows. No ChildrenCPUUsage yet.
  6. Add a pthread library for Windows in src/windows/pthread.{h,cc}. I got this library from http://locklessinc.com/articles/pthreads_on_windows/. I did minimal changes to make it compile (mostly fixing casts) and I separated it into .h and .cc files instead of having everything in the .h file (which caused linker errors). This library is distributed under a 3-clause BSD licence so I am assuming that using it is not problematic.
  7. Other than porting, I made two significant changes to the code of the benchmark library:
    • In BenchmarkFamilies::RemoveBenchmark I commented out some code which was changing a vector while iterating over it.
    • In State::FinishInterval I turned a CHECK into a warning because the wall time clock has poor accuracy on Windows 7.

@ckennelly
Copy link
Contributor

Can you rebase to get rid of the merge commits and the minor "cleanup" commits?

824324d seems to be doing multiple things other than porting the code to Windows.

@pleroy
Copy link
Contributor Author

pleroy commented Jun 8, 2014

Ok, I have squashed all the commits into a big one (e5efd2b), keeping the changes that I did to address some of your comments separate. It seems that unfortunately we have now lost quite a bit of the discussion (which doesn't seem unexpected since the history was rewritten and your comments were on individual commits).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't add typos, please. 😃

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in bdb0ba1.

@pphaneuf
Copy link
Contributor

pphaneuf commented Jun 8, 2014

Make sure to comment on the "files changed" tab of the pull request rather than on the commits.

When you comment on the commits, if there's any rebasing, it gets lost (or, at least, becomes hard to find). Also, depending on how you configured your GitHub notifications, those emails might go to personal rather than work email... 😃

@dmah42
Copy link
Member

dmah42 commented Jun 9, 2014

I've found that cmake generates perfectly good visual studio projects, and allows the setting of build flags and defines (such as OS_WINDOWS) perfectly well. I'd much rather continue using cmake as the de facto project description file as then all information is in one place.

This also removes the requirement for including port.h unless you need the symbols therein, which is much less error-prone.

.gitattributes Outdated
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should be the only one we care about, I believe.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 2579604.

@pleroy
Copy link
Contributor Author

pleroy commented Jun 9, 2014

@dominichamon: Sorry, I don't know cmake and I am not interested in learning it, so I'm not going there. I need a benchmark library on Windows, I'm happy to contribute the port, I realize it may not be perfect, I'm happy to address minor comments, but that's as much time as I am willing to invest in this.
Also, I don't think that it's very user-friendly to use cmake on Windows. As a Windows user I would expect to be able to use Visual Studio out of the box, without having to install prerequisites which try to make my system looks like *nix.

@d235j
Copy link

d235j commented Jul 7, 2014

cmake simply generates an MSVC project files based on the CMakeLists.txt, and is very much unlike the usual *nix build tools like automake and Make. CMake on Windows installs in C:\Program Files and has a rather nice GUI.

That said, are you doing anything special in the MSVC project? I made a minor change to the CMakeLists.txt (wrapping the three set(CMAKE_CXX_FLAGS) in if(NOT ${CMAKE_SYSTEM_NAME} MATCHES "Windows") ... endif() as the MSVC compiler doesn't understand those flags) and it happily generated an MSVC project file that compiled fine.

Of course, I don't know if the output is correct — it seems to build a .lib.

@pleroy
Copy link
Contributor Author

pleroy commented Jul 31, 2014

So @dominichamon, @ckennelly, where should we go with this pull request? I addressed quite a bit of the comments made in the original review and then the thread died out.

@dmah42
Copy link
Member

dmah42 commented Jul 31, 2014

@pleroy I just saw a flurry of activity here so I'll go through again. My biggest concern is adding the MSVC project and solution files as the whole point of using cmake is to avoid platform specific build and project management systems. I'm afraid this policy is firm for me.

If you don't want to learn it, then I ask that you drop those files from the pull request and i'll accept the build-specific changes. Someone else can then complete the work by ensuring cmake generates the correct project/solution. I don't have a windows machine so I can't take that on, I'm afraid.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should be brought in as a separate library in cmake rather than being copied into the project wholesale. We already do this for googletest using ExternalProject and that should work just fine here too.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't quite get this since I do know cmake, but from looking at what it does for googletest, I don't think it's applicable here, because I had to modify rather heavily the implementation that I found on the Interwebz. I guess there might be a better library around, but I would prefer to check in this version to get something that works sooner rather than later.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dominichamon, there is pthread win32 - we can have a look at working out the correct ExternalProject_Add options to build that correctly on Windows. The README is pretty thorough for that project.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with Dominic, I'd rather this exist outside of the project as well.

On a related note, is it absolutely necessary that this library is built alongside this project? Installing library prerequisites is somewhat common in the Linux world, so I'm not sure Windows should be different.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Having a look at the usage of pthread in the library, would a PR to port it to std::thread be acceptable?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, as long as it's supported back to around gcc 4.6. I think it is, though you still need to -lpthread for the older compilers, iirc. It also should be in MSVC 11?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll have a look at gcc support. I'll write it in a way that the benchmark::thread is selected from std::thread, boost::thread and then just::thread as they all provide the same semantics (for our purposes). I'll do it using the same detection that the regex back end enjoys.

Apparently there are some bugs in the MSVC11 implementation.

@pleroy
Copy link
Contributor Author

pleroy commented Aug 1, 2014

@dominichamon: I understand the desire to use a unified build system, and I'd be fine with removing the MSVC files from this pull request. However, I still think that the MSVC files generated by cmake should ultimately be in the repository, because otherwise you are asking every Windows user to install cmake. That's AFAICT the approach adopted by gtest and glog.

@mattyclarkson
Copy link
Contributor

@pleroy , the problem with adding MSVS files to the project is it supports just a small subset of users - what version of MSVS files do we commit? VS2013? VS2012 with clang? VS2010 with clang? What do we do when VS2014 comes around? I actually have gcc integrated into MSVS - the solution files won't help me with that.

It is the same requirement on any OS, I have to install cmake on my Arch, Ubuntu and Fedora boxes to build this library with Eclipse or make. On MacOSX people may want to build with XCode or make, but they have to have cmake to do it. It would be the same to add XCode or Eclipse IDE configuration files to the project. That would allow IDE users on MacOSX and Linux to not have to install cmake but then you end up with combinatorial explosion of IDE solutions.

I would happily write you a Windows batch script that downloaded the latest cmake binaries from their website (no install) and generated a MSVS solution file by double clicking on the .bat if you feel that is needed for Windows users. However, aside from the gtest and glog examples you provide above I haven't personally come across any CMake projects that include the solution files and find it a clean solution to building projects.

This is obviously my two cents, I'm not bashing on your opinion and respect that you would like to have MSVS solution for convenience. Just trying to provide another view 😃

@ckennelly
Copy link
Contributor

Before this pull request is merged, can it be rebased against master? e5efd2b (add MSVC files) and 7b145d7 (remove MSVC files) are redundant (and don't need to become part of the project's permanent history).

@pleroy
Copy link
Contributor Author

pleroy commented Aug 4, 2014

@mattyclarkson: "Simple things should be simple, complex things should be possible." Provide the necessary files to run with the latest VS (2013 at this point) so that people with the most basic requirements can get started in 30 minutes. Those who want to use, say, VS 2008 with clang will either use cmake or create their project setup themselves. If you are making things complicated for the simplest use case you'll lose 80% of the users.

@pphaneuf
Copy link
Contributor

pphaneuf commented Aug 4, 2014

If you provide projects for specific environment, I would suggest putting them in a contrib directory, say, to make it clear that they are only an unsupported best-effort thing?

@izaid
Copy link

izaid commented Jan 6, 2015

Can I ask what the status is on this? What needs to be done to make it work?

@mattyclarkson
Copy link
Contributor

@izaid, I've done some porting to MinGW over at #64. That branch just needs a little bit more to get working on VS. Haven't had any time to do it recently though.

@EricWF
Copy link
Contributor

EricWF commented Mar 18, 2015

With the recent and large scale changes to just about all the code I don't think this patch is going to merge in any meaningful way.

Is there anybody that is willing to continue on with this work?

@mattyclarkson
Copy link
Contributor

@EricWF, I'm keen to get the benchmarking library to work on Windows. I wrote a thinner MinGW port branch #64 but have been swamped with other stuff. I can try to have a look soon. I'll have to get reacquainted with the recent changes that have gone in though. Probably best to close both and create an issue for Windows support and wait for someone to pick it up?

@EricWF
Copy link
Contributor

EricWF commented Mar 18, 2015

The MinGW port in #64 looks reasonable. Lets try and get that in soon.

@pleroy
Copy link
Contributor Author

pleroy commented Mar 18, 2015

@EricWF, yeah, it seems that if PRs are left unanswered for 6 months they become impossible to merge. I'll close this and keep using my own fork at pleroy/benchmark. I am personally not interested in MinGW.

@pleroy pleroy closed this Mar 18, 2015
@mattyclarkson
Copy link
Contributor

@pleroy, what version of Visual Studio are you using? I can have a look at adding support now that you can get VS for free.

@pleroy
Copy link
Contributor Author

pleroy commented Mar 20, 2015

@mattyclarkson: I am using VS 2013. It's pretty much the only version that makes sense anyway for this code because previous versions didn't have C++11 support.

@mattyclarkson
Copy link
Contributor

@pleroy, I'll have a look when I do the MinGW port - hopefully there shouldn't be much that VS needs to get going.

@mattyclarkson
Copy link
Contributor

@pleroy, I had a look at getting MSVC up and running on the project. There's not much that needs to be done with regards to polyfilling the unix functions. Only gettimeofday really needs to be implemented and the correct headers. I've given up as I can't get on with the MSVC compilers implementation of the C++ standard and the warnings that it throws up. I'll have another look when VS 2015 is out and the compiler has moved along a bit. TBH, by then clang will be probably be a better alternative anyway.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants