Index: ps/trunk/source/lib/status.h =================================================================== --- ps/trunk/source/lib/status.h (revision 9544) +++ ps/trunk/source/lib/status.h (revision 9545) @@ -1,449 +1,449 @@ /* Copyright (c) 2011 Wildfire Games * * Permission is hereby granted, free of charge, to any person obtaining * a copy of this software and associated documentation files (the * "Software"), to deal in the Software without restriction, including * without limitation the rights to use, copy, modify, merge, publish, * distribute, sublicense, and/or sell copies of the Software, and to * permit persons to whom the Software is furnished to do so, subject to * the following conditions: * * The above copyright notice and this permission notice shall be included * in all copies or substantial portions of the Software. * * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. * IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY * CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, * TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE * SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. */ /* * error handling system: defines status codes, translates them to/from * other schemes (e.g. errno), associates them with descriptive text, * simplifies propagating errors / checking if functions failed. */ /** Error handling system Why Error Codes? ---------------- To convey information about what failed, the alternatives are unique integral codes and direct pointers to descriptive text. Both occupy the same amount of space, but codes are easier to internationalize. Method of Propagating Errors ---------------------------- When a low-level function has failed, this must be conveyed to the higher-level application logic across several functions on the call stack. There are two alternatives: 1) check at each call site whether a function failed; if so, return to the caller. 2) throw an exception. We will discuss the advantages and disadvantages of exceptions, which are the opposites of call site checking. - performance: they shouldn't be used in time-critical code. - predictability: exceptions can come up almost anywhere, so it is hard to say what execution path will be taken. - interoperability: not compatible with other languages. + readability: cleans up code by separating application logic and error handling. however, this is also a disadvantage because it may be difficult to see at a glance if a piece of code does error checking at all. + visibility: errors are more likely to be seen than relying on callers to check return codes; less reliant on discipline. Both have their place. Our recommendation is to throw error code exceptions when checking call sites and propagating errors becomes tedious. However, inter-module boundaries should always return error codes for interoperability with other languages. Simplifying Call-Site Checking ------------------------------ As mentioned above, this approach requires discipline. We provide "enforcer" macros to simplify this task by propagating errors to the calling function. Consider the following example: Status status = doWork(); if(status != INFO::OK) return status; This can be replaced by: RETURN_STATUS_IF_ERR(doWork()); This provides a visible sign that the code handles errors but reduces clutter. When to warn the user? ---------------------- When a function fails, there are 2 places we can raise a warning: as soon as the error condition is known, or higher on the call stack. We prefer the former because it is easier to ensure that all possible return paths have been covered: search for all "return ERR::*" or "return StatusFrom*" that are not followed by a "// NOWARN" comment. The latter approach also risks multiple warnings along the call stack for the same error. Note the special case of "validator" functions that e.g. verify the state of an object: we now discuss pros/cons of just returning errors without warning, and having their callers take care of that. + they typically have many return paths (-> increased code size) - this is balanced by validators that have many call sites. - we want all return statements wrapped for consistency and easily checking if any were forgotten - adding // NOWARN to each validator return statement would be tedious. - there is no advantage to checking at the call site; the call stack indicates which caller of the validator failed anyway. Validator functions should therefore also use WARN_RETURN. Numbering Scheme ---------------- Each module header defines its own error codes to avoid a full rebuild whenever a new code is added. Error codes start at -100000 (warnings are positive, but the corresponding negative value should not be used to avoid confusion). This scheme avoids collisions with all other known error codes. Each header gets 100 possible values; the tens value may be used to denote groups within that header. The subsystem is denoted by the ten-thousands digit: 0 general 1 file 2 res (resource management) 3 sysdep (system-dependent) 4 win (Windows-specific) To summarize: +/-1SHHCC (S=subsystem, HH=header, CC=code number) 10 general 00CC misc 03CC path 04CC debug 05CC debug_stl 06CC secure_crt 07CC wchar 11 file 01CC vfs 03CC file 04CC archive 12 res 01CC tex 02CC ogl_shader 13 sysdep 00CC cpu 01CC os_cpu 14 win 00CC whrt **/ #ifndef INCLUDED_STATUS #define INCLUDED_STATUS #include "lib/lib_api.h" // an integral type allows defining error codes in separate headers, // but is not as type-safe as an enum. use Lint's 'strong type' checking // to catch errors such as Status Func() { return 1; }. // this must be i64 because some functions may multiplex Status with // file offsets/sizes in their return value. typedef i64 Status; // associates a status code with a description [and errno_equivalent]. struct StatusDefinition // POD { Status status; // typically a string literal; must remain valid until end of program. const wchar_t* description; // omit initializer (or initialize to 0) if there is no errno equivalent. int errno_equivalent; }; // retrieving description and errno_equivalent requires going through all // StatusDefinition instances. we avoid dynamic memory allocation (which // is problematic because status codes may be needed before _cinit) by // organizing them into a linked list, with nodes residing in static storage. // since modules may introduce many status codes, they are stored in an // array, aka "bucket", which includes a link to the next bucket. // initialized via STATUS_ADD_DEFINITIONS; opaque. struct StatusDefinitionBucket // POD { const StatusDefinition* definitions; size_t numDefinitions; StatusDefinitionBucket* next; }; /** * (called via STATUS_ADD_DEFINITIONS) * * @param bucket is being added; its definitions and numDefinitions must * already be initialized. * @return previous bucket in list, suitable for initializing bucket->next. * * (this function must be callable as a static initializer; initializing * next avoids the need for a separate dummy variable) **/ LIB_API StatusDefinitionBucket* StatusAddDefinitions(StatusDefinitionBucket* bucket); /** * add a module's array of StatusDefinition to the list. * typically invoked at file scope. * @param definitions name (identifier) of the array **/ -#define STATUS_ADD_DEFINITIONS(definitions) static StatusDefinitionBucket definitions##_bucket = { definitions, ARRAY_SIZE(definitions), StatusAddDefinitions(&definitions##_bucket) }; +#define STATUS_ADD_DEFINITIONS(definitions) static StatusDefinitionBucket definitions##_bucket = { definitions, ARRAY_SIZE(definitions), StatusAddDefinitions(&definitions##_bucket) } /** * generate textual description of a Status. * * @param buf destination buffer (allows generating strings with * the code's numerical value if no definition is found) * @param max_chars size of buffer [characters] * @return buf (allows using this function in expressions) **/ LIB_API wchar_t* StatusDescription(Status status, wchar_t* buf, size_t max_chars); /** * @return the errno equivalent of a Status. * * used in wposix - underlying functions return Status but must be * translated to errno at e.g. the mmap interface level. higher-level code * that calls mmap will in turn convert back to Status. **/ extern int ErrnoFromStatus(Status status); /** * @return Status equivalent of errno, or ERR::FAIL if there's no equivalent. * * NB: reset errno to 0 before calling POSIX functions to avoid confusion * with previous errors. **/ extern Status StatusFromErrno(); // note: other conversion routines (e.g. to/from Win32) are implemented in // the corresponding modules to keep this header portable. //----------------------------------------------------------------------------- // propagation macros // warn and return a status. use when an error is first detected to // begin propagating it to callers. #define WARN_RETURN(status)\ do\ {\ DEBUG_WARN_ERR(status);\ return status;\ }\ while(0) // warn if expression is negative, i.e. an error. // (this macro is more convenient than ENSURE) #define WARN_IF_ERR(expression)\ do\ {\ const Status status_ = (expression);\ if(status_ < 0)\ DEBUG_WARN_ERR(status_);\ }\ while(0) // return expression if it is negative, i.e. pass on errors to // the caller. use when failures are common/expected. #define RETURN_STATUS_IF_ERR(expression)\ do\ {\ const Status status_ = (expression);\ if(status_ < 0)\ return status_;\ }\ while(0) // warn and return expression if it is negative. // use if a function doesn't raise warnings when it returns errors. #define WARN_RETURN_STATUS_IF_ERR(expression)\ do\ {\ const Status status_ = (expression);\ if(status_ < 0)\ {\ DEBUG_WARN_ERR(status_);\ return status_;\ }\ }\ while(0) // warn and throw expression if it is negative. use to propagate // errors from within constructors. #define THROW_STATUS_IF_ERR(expression)\ do\ {\ const Status status_ = (expression);\ if(status_ < 0)\ {\ DEBUG_WARN_ERR(status_);\ throw status_;\ }\ }\ while(0) // return 0 if expression is negative. use in functions that return pointers. #define RETURN_0_IF_ERR(expression)\ do\ {\ const Status status_ = (expression);\ if(status_ < 0)\ return 0;\ }\ while(0) // return expression if it evaluates to something other than // INFO::CONTINUE. use when invoking callbacks. #define RETURN_IF_NOT_CONTINUE(expression)\ do\ {\ const Status status_ = (expression);\ if(status_ != INFO::CONTINUE)\ return status_;\ }\ while(0) // warn if expression is false, i.e. zero. #define WARN_IF_FALSE(expression)\ do\ {\ if(!(expression))\ debug_warn(L"FYI: WARN_IF_FALSE reports that a function failed. Feel free to ignore or suppress this warning.");\ }\ while(0) // warn and return 0 if expression is false, i.e. zero. #define WARN_RETURN_0_IF_FALSE(expression)\ do\ {\ if(!(expression))\ {\ debug_warn(L"FYI: WARN_RETURN_0_IF_FALSE reports that a function failed. Feel free to ignore or suppress this warning.");\ return 0;\ }\ }\ while(0) //----------------------------------------------------------------------------- // shared status code definitions namespace INFO { const Status OK = 0; // note: these values are > 100 to allow multiplexing them with // coroutine return values, which return completion percentage. // the function (usually a callback) would like to be called again. const Status CONTINUE = +100000; // notify caller that nothing was done. const Status SKIPPED = +100001; // function is incapable of doing the requested task with the given inputs. // this implies SKIPPED, but also conveys a bit more information. const Status CANNOT_HANDLE = +100002; // function is meant to be called repeatedly, and now indicates that // all jobs are complete. const Status ALL_COMPLETE = +100003; } // namespace INFO namespace ERR { const Status FAIL = -1; // unknown failure // general const Status LOGIC = -100010; const Status EXCEPTION = -100011; const Status TIMED_OUT = -100012; const Status REENTERED = -100013; const Status CORRUPTED = -100014; const Status ABORTED = -100015; // invalid values (usually function arguments) const Status INVALID_ALIGNMENT = -100020; const Status INVALID_OFFSET = -100021; const Status INVALID_HANDLE = -100022; const Status INVALID_POINTER = -100023; const Status INVALID_SIZE = -100024; const Status INVALID_FLAG = -100025; const Status INVALID_PARAM = -100026; const Status INVALID_VERSION = -100027; // system limitations const Status AGAIN = -100030; const Status LIMIT = -100031; const Status NOT_SUPPORTED = -100032; const Status NO_MEM = -100033; // these are for cases where we just want a distinct value to display and // a symbolic name + string would be overkill (e.g. the various // test cases in a validate() call). they are shared between multiple // functions; when something fails, the stack trace will show in which // one it was => these errors are unambiguous. // there are 3 tiers - 1..9 are used in most functions, 11..19 are // used in a function that calls another validator and 21..29 are // for for functions that call 2 other validators (this avoids // ambiguity as to which error actually happened where) const Status _1 = -100101; const Status _2 = -100102; const Status _3 = -100103; const Status _4 = -100104; const Status _5 = -100105; const Status _6 = -100106; const Status _7 = -100107; const Status _8 = -100108; const Status _9 = -100109; const Status _11 = -100111; const Status _12 = -100112; const Status _13 = -100113; const Status _14 = -100114; const Status _15 = -100115; const Status _16 = -100116; const Status _17 = -100117; const Status _18 = -100118; const Status _19 = -100119; const Status _21 = -100121; const Status _22 = -100122; const Status _23 = -100123; const Status _24 = -100124; const Status _25 = -100125; const Status _26 = -100126; const Status _27 = -100127; const Status _28 = -100128; const Status _29 = -100129; } // namespace ERR #endif // #ifndef INCLUDED_STATUS Index: ps/trunk/source/lib/allocators/aligned_allocator.h =================================================================== --- ps/trunk/source/lib/allocators/aligned_allocator.h (nonexistent) +++ ps/trunk/source/lib/allocators/aligned_allocator.h (revision 9545) @@ -0,0 +1,147 @@ +/* Copyright (c) 2010 Wildfire Games + * + * Permission is hereby granted, free of charge, to any person obtaining + * a copy of this software and associated documentation files (the + * "Software"), to deal in the Software without restriction, including + * without limitation the rights to use, copy, modify, merge, publish, + * distribute, sublicense, and/or sell copies of the Software, and to + * permit persons to whom the Software is furnished to do so, subject to + * the following conditions: + * + * The above copyright notice and this permission notice shall be included + * in all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. + * IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY + * CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, + * TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE + * SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + */ + +/* + * STL allocator for aligned memory + */ + +#ifndef ALIGNED_ALLOCATOR +#define ALIGNED_ALLOCATOR + +#include "lib/bits.h" // round_up +#include "lib/sysdep/arch/x86_x64/cache.h" +#include "lib/sysdep/rtl.h" // rtl_AllocateAligned + + +/** + * stateless STL allocator that aligns elements to the L1 cache line size. + * + * note: the alignment is hard-coded to avoid any allocator state. + * this avoids portability problems, which is important since allocators + * are rather poorly specified. + * + * references: + * http://www.tantalon.com/pete/customallocators.ppt + * http://www.flipcode.com/archives/Aligned_Block_Allocation.shtml + * http://www.josuttis.com/cppcode/allocator.html + * + * derived from code that bears the following copyright notice: + * (C) Copyright Nicolai M. Josuttis 1999. + * Permission to copy, use, modify, sell and distribute this software + * is granted provided this copyright notice appears in all copies. + * This software is provided "as is" without express or implied + * warranty, and with no claim as to its suitability for any purpose. + **/ +template +class AlignedAllocator +{ +public: + // type definitions + typedef T value_type; + typedef T* pointer; + typedef const T* const_pointer; + typedef T& reference; + typedef const T& const_reference; + typedef std::size_t size_type; + typedef std::ptrdiff_t difference_type; + + // rebind allocator to type U + template + struct rebind + { + typedef AlignedAllocator other; + }; + + pointer address(reference value) const + { + return &value; + } + + const_pointer address(const_reference value) const + { + return &value; + } + + AlignedAllocator() throw() + { + } + + AlignedAllocator(const AlignedAllocator&) throw() + { + } + + template + AlignedAllocator (const AlignedAllocator&) throw() + { + } + + ~AlignedAllocator() throw() + { + } + + size_type max_size() const throw() + { + // maximum number of *elements* that can be allocated + return std::numeric_limits::max() / sizeof(T); + } + + // allocate uninitialized storage + pointer allocate(size_type numElements) + { + const size_type alignment = x86_x64_Caches(L1D)->entrySize; + const size_type elementSize = round_up(sizeof(T), alignment); + const size_type size = numElements * elementSize; + pointer p = (pointer)rtl_AllocateAligned(size, alignment); + return p; + } + + // deallocate storage of elements that have been destroyed + void deallocate(pointer p, size_type UNUSED(num)) + { + rtl_FreeAligned((void*)p); + } + + void construct(pointer p, const T& value) + { + new((void*)p) T(value); + } + + void destroy(pointer p) + { + p->~T(); + } +}; + +// indicate that all specializations of this allocator are interchangeable +template +bool operator==(const AlignedAllocator&, const AlignedAllocator&) throw() +{ + return true; +} + +template +bool operator!=(const AlignedAllocator&, const AlignedAllocator&) throw() +{ + return false; +} + +#endif // #ifndef ALIGNED_ALLOCATOR Property changes on: ps/trunk/source/lib/allocators/aligned_allocator.h ___________________________________________________________________ Added: svn:eol-style ## -0,0 +1 ## +native \ No newline at end of property Index: ps/trunk/source/lib/status.cpp =================================================================== --- ps/trunk/source/lib/status.cpp (revision 9544) +++ ps/trunk/source/lib/status.cpp (revision 9545) @@ -1,179 +1,178 @@ /* Copyright (c) 2011 Wildfire Games * * Permission is hereby granted, free of charge, to any person obtaining * a copy of this software and associated documentation files (the * "Software"), to deal in the Software without restriction, including * without limitation the rights to use, copy, modify, merge, publish, * distribute, sublicense, and/or sell copies of the Software, and to * permit persons to whom the Software is furnished to do so, subject to * the following conditions: * * The above copyright notice and this permission notice shall be included * in all copies or substantial portions of the Software. * * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. * IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY * CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, * TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE * SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. */ /* * error handling system: defines status codes, translates them to/from * other schemes (e.g. errno), associates them with descriptive text, * simplifies propagating errors / checking if functions failed. */ #include "precompiled.h" #include "lib/status.h" #include #include #include "lib/posix/posix_errno.h" static StatusDefinitionBucket* buckets; StatusDefinitionBucket* StatusAddDefinitions(StatusDefinitionBucket* bucket) { // insert at front of list StatusDefinitionBucket* next = buckets; buckets = bucket; return next; } static const StatusDefinition* DefinitionFromStatus(Status status) { for(const StatusDefinitionBucket* bucket = buckets; bucket; bucket = bucket->next) { for(size_t i = 0; i < bucket->numDefinitions; i++) { if(bucket->definitions[i].status == status) return &bucket->definitions[i]; } } return 0; } static const StatusDefinition* DefinitionFromErrno(int errno_equivalent) { for(const StatusDefinitionBucket* bucket = buckets; bucket; bucket = bucket->next) { for(size_t i = 0; i < bucket->numDefinitions; i++) { if(bucket->definitions[i].errno_equivalent == errno_equivalent) return &bucket->definitions[i]; } } return 0; } wchar_t* StatusDescription(Status status, wchar_t* buf, size_t max_chars) { const StatusDefinition* def = DefinitionFromStatus(status); if(def) { wcscpy_s(buf, max_chars, def->description); return buf; } swprintf_s(buf, max_chars, L"Unknown error (%lld, 0x%llX)", status, status); return buf; } int ErrnoFromStatus(Status status) { const StatusDefinition* def = DefinitionFromStatus(status); if(def && def->errno_equivalent != 0) return def->errno_equivalent; // the set of errnos in wposix.h doesn't have an "unknown error". // we use this one as a default because it's not expected to come up often. return EPERM; } Status StatusFromErrno() { if(errno == 0) - return ERR::FAIL; - + return INFO::OK; const StatusDefinition* def = DefinitionFromErrno(errno); return def? def->status : ERR::FAIL; } //----------------------------------------------------------------------------- static const StatusDefinition statusDefs[] = { // INFO::OK doesn't really need a string because calling StatusDescription(0) // should never happen, but we'll play it safe. -{ INFO::OK, L"(but return value was 0 which indicates success)" }, +{ INFO::OK, L"No error reported here" }, { ERR::FAIL, L"Function failed (no details available)" }, { INFO::CONTINUE, L"Continue (not an error)" }, { INFO::SKIPPED, L"Skipped (not an error)" }, { INFO::CANNOT_HANDLE, L"Cannot handle (not an error)" }, { INFO::ALL_COMPLETE, L"All complete (not an error)" }, { ERR::LOGIC, L"Logic error in code" }, { ERR::EXCEPTION, L"Caught an exception" }, { ERR::TIMED_OUT, L"Timed out" }, { ERR::REENTERED, L"Single-call function was reentered" }, { ERR::CORRUPTED, L"File/memory data is corrupted" }, { ERR::ABORTED, L"Operation aborted" }, { ERR::INVALID_ALIGNMENT, L"Invalid alignment", EINVAL }, { ERR::INVALID_OFFSET, L"Invalid offset", EINVAL }, { ERR::INVALID_HANDLE, L"Invalid handle", EINVAL }, { ERR::INVALID_POINTER, L"Invalid pointer", EINVAL }, { ERR::INVALID_SIZE, L"Invalid size", EINVAL }, { ERR::INVALID_FLAG, L"Invalid flag", EINVAL }, { ERR::INVALID_PARAM, L"Invalid parameter", EINVAL }, { ERR::INVALID_VERSION, L"Invalid version", EINVAL }, { ERR::AGAIN, L"Try again later", EAGAIN }, { ERR::LIMIT, L"Fixed limit exceeded", E2BIG }, { ERR::NOT_SUPPORTED, L"Function not supported", ENOSYS }, { ERR::NO_MEM, L"Not enough memory", ENOMEM}, { ERR::_1, L"Case 1" }, { ERR::_2, L"Case 2" }, { ERR::_3, L"Case 3" }, { ERR::_4, L"Case 4" }, { ERR::_5, L"Case 5" }, { ERR::_6, L"Case 6" }, { ERR::_7, L"Case 7" }, { ERR::_8, L"Case 8" }, { ERR::_9, L"Case 9" }, { ERR::_11, L"Case 11" }, { ERR::_12, L"Case 12" }, { ERR::_13, L"Case 13" }, { ERR::_14, L"Case 14" }, { ERR::_15, L"Case 15" }, { ERR::_16, L"Case 16" }, { ERR::_17, L"Case 17" }, { ERR::_18, L"Case 18" }, { ERR::_19, L"Case 19" }, { ERR::_21, L"Case 21" }, { ERR::_22, L"Case 22" }, { ERR::_23, L"Case 23" }, { ERR::_24, L"Case 24" }, { ERR::_25, L"Case 25" }, { ERR::_26, L"Case 26" }, { ERR::_27, L"Case 27" }, { ERR::_28, L"Case 28" }, { ERR::_29, L"Case 29" } }; -STATUS_ADD_DEFINITIONS(statusDefs) +STATUS_ADD_DEFINITIONS(statusDefs); Index: ps/trunk/source/lib/debug.cpp =================================================================== --- ps/trunk/source/lib/debug.cpp (revision 9544) +++ ps/trunk/source/lib/debug.cpp (revision 9545) @@ -1,552 +1,554 @@ /* Copyright (c) 2010 Wildfire Games * * Permission is hereby granted, free of charge, to any person obtaining * a copy of this software and associated documentation files (the * "Software"), to deal in the Software without restriction, including * without limitation the rights to use, copy, modify, merge, publish, * distribute, sublicense, and/or sell copies of the Software, and to * permit persons to whom the Software is furnished to do so, subject to * the following conditions: * * The above copyright notice and this permission notice shall be included * in all copies or substantial portions of the Software. * * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. * IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY * CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, * TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE * SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. */ /* * platform-independent debug support code. */ #include "precompiled.h" #include "lib/debug.h" #include #include #include #include "lib/alignment.h" #include "lib/app_hooks.h" #include "lib/allocators/page_aligned.h" #include "lib/fnv_hash.h" #include "lib/sysdep/cpu.h" // cpu_CAS #include "lib/sysdep/sysdep.h" #if OS_WIN # include "lib/sysdep/os/win/wdbg_heap.h" #endif static const StatusDefinition debugStatusDefinitions[] = { { ERR::SYM_NO_STACK_FRAMES_FOUND, L"No stack frames found" }, { ERR::SYM_UNRETRIEVABLE_STATIC, L"Value unretrievable (stored in external module)" }, { ERR::SYM_UNRETRIEVABLE, L"Value unretrievable" }, { ERR::SYM_TYPE_INFO_UNAVAILABLE, L"Error getting type_info" }, { ERR::SYM_INTERNAL_ERROR, L"Exception raised while processing a symbol" }, { ERR::SYM_UNSUPPORTED, L"Symbol type not (fully) supported" }, { ERR::SYM_CHILD_NOT_FOUND, L"Symbol does not have the given child" }, { ERR::SYM_NESTING_LIMIT, L"Symbol nesting too deep or infinite recursion" }, { ERR::SYM_SINGLE_SYMBOL_LIMIT, L"Symbol has produced too much output" }, { INFO::SYM_SUPPRESS_OUTPUT, L"Symbol was suppressed" } }; STATUS_ADD_DEFINITIONS(debugStatusDefinitions); // need to shoehorn printf-style variable params into // the OutputDebugString call. // - don't want to split into multiple calls - would add newlines to output. // - fixing Win32 _vsnprintf to return # characters that would be written, // as required by C99, looks difficult and unnecessary. if any other code // needs that, implement GNU vasprintf. // - fixed size buffers aren't nice, but much simpler than vasprintf-style // allocate+expand_until_it_fits. these calls are for quick debug output, // not loads of data, anyway. // rationale: static data instead of std::set to allow setting at any time. // we store FNV hash of tag strings for fast comparison; collisions are // extremely unlikely and can only result in displaying more/less text. static const size_t MAX_TAGS = 20; static u32 tags[MAX_TAGS]; static size_t num_tags; void debug_filter_add(const wchar_t* tag) { const u32 hash = fnv_hash(tag, wcslen(tag)*sizeof(tag[0])); // make sure it isn't already in the list for(size_t i = 0; i < MAX_TAGS; i++) if(tags[i] == hash) return; // too many already? if(num_tags == MAX_TAGS) { DEBUG_WARN_ERR(ERR::LOGIC); // increase MAX_TAGS return; } tags[num_tags++] = hash; } void debug_filter_remove(const wchar_t* tag) { const u32 hash = fnv_hash(tag, wcslen(tag)*sizeof(tag[0])); for(size_t i = 0; i < MAX_TAGS; i++) { if(tags[i] == hash) // found it { // replace with last element (avoid holes) tags[i] = tags[MAX_TAGS-1]; num_tags--; // can only happen once, so we're done. return; } } } void debug_filter_clear() { std::fill(tags, tags+MAX_TAGS, 0); } bool debug_filter_allows(const wchar_t* text) { size_t i; for(i = 0; ; i++) { // no | found => no tag => should always be displayed if(text[i] == ' ' || text[i] == '\0') return true; if(text[i] == '|' && i != 0) break; } const u32 hash = fnv_hash(text, i*sizeof(text[0])); // check if entry allowing this tag is found for(i = 0; i < MAX_TAGS; i++) if(tags[i] == hash) return true; return false; } #undef debug_printf // allowing #defining it out void debug_printf(const wchar_t* fmt, ...) { wchar_t buf[4096]; va_list ap; va_start(ap, fmt); const int numChars = vswprintf_s(buf, ARRAY_SIZE(buf), fmt, ap); ENSURE(numChars >= 0); va_end(ap); if(debug_filter_allows(buf)) debug_puts(buf); } //----------------------------------------------------------------------------- Status debug_WriteCrashlog(const wchar_t* text) { // (avoid infinite recursion and/or reentering this function if it // fails/reports an error) enum State { IDLE, BUSY, FAILED }; // note: the initial state is IDLE. we rely on zero-init because // initializing local static objects from constants may happen when // this is first called, which isn't thread-safe. (see C++ 6.7.4) cassert(IDLE == 0); static volatile intptr_t state; if(!cpu_CAS(&state, IDLE, BUSY)) return ERR::REENTERED; // NOWARN OsPath pathname = ah_get_log_dir()/"crashlog.txt"; FILE* f = sys_OpenFile(pathname, "w"); if(!f) { state = FAILED; // must come before DEBUG_DISPLAY_ERROR DEBUG_DISPLAY_ERROR(L"Unable to open crashlog.txt for writing (please ensure the log directory is writable)"); return ERR::FAIL; // NOWARN (the above text is more helpful than a generic error code) } fputwc(0xFEFF, f); // BOM fwprintf(f, L"%ls\n", text); fwprintf(f, L"\n\n====================================\n\n"); // allow user to bundle whatever information they want ah_bundle_logs(f); fclose(f); state = IDLE; return INFO::OK; } //----------------------------------------------------------------------------- // error message //----------------------------------------------------------------------------- // (NB: this may appear obscene, but deep stack traces have been // observed to take up > 256 KiB) static const size_t messageSize = 512*KiB; void debug_FreeErrorMessage(ErrorMessageMem* emm) { page_aligned_free(emm->pa_mem, messageSize); } // a stream with printf-style varargs and the possibility of // writing directly to the output buffer. class PrintfWriter { public: PrintfWriter(wchar_t* buf, size_t maxChars) : m_pos(buf), m_charsLeft(maxChars) { } bool operator()(const wchar_t* fmt, ...) WPRINTF_ARGS(2) { va_list ap; va_start(ap, fmt); const int len = vswprintf_s(m_pos, m_charsLeft, fmt, ap); va_end(ap); if(len < 0) return false; m_pos += len; m_charsLeft -= len; return true; } wchar_t* Position() const { return m_pos; } size_t CharsLeft() const { return m_charsLeft; } void CountAddedChars() { const size_t len = wcslen(m_pos); m_pos += len; m_charsLeft -= len; } private: wchar_t* m_pos; size_t m_charsLeft; }; // split out of debug_DisplayError because it's used by the self-test. const wchar_t* debug_BuildErrorMessage( const wchar_t* description, const wchar_t* filename, int line, const char* func, void* context, const wchar_t* lastFuncToSkip, ErrorMessageMem* emm) { + // retrieve errno (might be relevant) before doing anything else + // that might overwrite it. + wchar_t description_buf[100] = L"?"; + wchar_t os_error[100] = L"?"; + Status errno_equiv = StatusFromErrno(); // NOWARN + if(errno_equiv != ERR::FAIL) // meaningful translation + StatusDescription(errno_equiv, description_buf, ARRAY_SIZE(description_buf)); + sys_StatusDescription(0, os_error, ARRAY_SIZE(os_error)); + // rationale: see ErrorMessageMem emm->pa_mem = page_aligned_alloc(messageSize); wchar_t* const buf = (wchar_t*)emm->pa_mem; if(!buf) return L"(insufficient memory to generate error message)"; PrintfWriter writer(buf, messageSize / sizeof(wchar_t)); // header if(!writer( L"%ls\r\n" L"Location: %ls:%d (%hs)\r\n" L"\r\n" L"Call stack:\r\n" L"\r\n", description, filename, line, func )) { fail: return L"(error while formatting error message)"; } // append stack trace Status ret = debug_DumpStack(writer.Position(), writer.CharsLeft(), context, lastFuncToSkip); if(ret == ERR::REENTERED) { if(!writer( L"While generating an error report, we encountered a second " L"problem. Please be sure to report both this and the subsequent " L"error messages." )) goto fail; } else if(ret != INFO::OK) { - wchar_t description_buf[100] = {'?'}; + wchar_t error_buf[100] = {'?'}; if(!writer( L"(error while dumping stack: %ls)", - StatusDescription(ret, description_buf, ARRAY_SIZE(description_buf)) + StatusDescription(ret, error_buf, ARRAY_SIZE(error_buf)) )) goto fail; } else // success { writer.CountAddedChars(); } - // append OS error (just in case it happens to be relevant - - // it's usually still set from unrelated operations) - wchar_t description_buf[100] = L"?"; - Status errno_equiv = StatusFromErrno(); // NOWARN - if(errno_equiv != ERR::FAIL) // meaningful translation - StatusDescription(errno_equiv, description_buf, ARRAY_SIZE(description_buf)); - wchar_t os_error[100] = L"?"; - sys_StatusDescription(0, os_error, ARRAY_SIZE(os_error)); + // append errno if(!writer( L"\r\n" L"errno = %d (%ls)\r\n" L"OS error = %ls\r\n", errno, description_buf, os_error )) goto fail; return buf; } //----------------------------------------------------------------------------- // display error messages //----------------------------------------------------------------------------- // translates and displays the given strings in a dialog. // this is typically only used when debug_DisplayError has failed or // is unavailable because that function is much more capable. // implemented via sys_display_msg; see documentation there. void debug_DisplayMessage(const wchar_t* caption, const wchar_t* msg) { sys_display_msg(ah_translate(caption), ah_translate(msg)); } // when an error has come up and user clicks Exit, we don't want any further // errors (e.g. caused by atexit handlers) to come up, possibly causing an // infinite loop. hiding errors isn't good, but we assume that whoever clicked // exit really doesn't want to see any more messages. static atomic_bool isExiting; // this logic is applicable to any type of error. special cases such as // suppressing certain expected WARN_ERRs are done there. static bool ShouldSuppressError(atomic_bool* suppress) { if(isExiting) return true; if(!suppress) return false; if(*suppress == DEBUG_SUPPRESS) return true; return false; } static ErrorReactionInternal CallDisplayError(const wchar_t* text, size_t flags) { // first try app hook implementation ErrorReactionInternal er = ah_display_error(text, flags); // .. it's only a stub: default to normal implementation if(er == ERI_NOT_IMPLEMENTED) er = sys_display_error(text, flags); return er; } static ErrorReaction PerformErrorReaction(ErrorReactionInternal er, size_t flags, atomic_bool* suppress) { const bool shouldHandleBreak = (flags & DE_MANUAL_BREAK) == 0; switch(er) { case ERI_CONTINUE: return ER_CONTINUE; case ERI_BREAK: // handle "break" request unless the caller wants to (doing so here // instead of within the dlgproc yields a correct call stack) if(shouldHandleBreak) { debug_break(); return ER_CONTINUE; } else return ER_BREAK; case ERI_SUPPRESS: (void)cpu_CAS(suppress, 0, DEBUG_SUPPRESS); return ER_CONTINUE; case ERI_EXIT: isExiting = 1; // see declaration COMPILER_FENCE; #if OS_WIN // prevent (slow) heap reporting since we're exiting abnormally and // thus probably leaking like a sieve. wdbg_heap_Enable(false); #endif exit(EXIT_FAILURE); case ERI_NOT_IMPLEMENTED: default: debug_break(); // not expected to be reached return ER_CONTINUE; } } ErrorReaction debug_DisplayError(const wchar_t* description, size_t flags, void* context, const wchar_t* lastFuncToSkip, const wchar_t* pathname, int line, const char* func, atomic_bool* suppress) { // "suppressing" this error means doing nothing and returning ER_CONTINUE. if(ShouldSuppressError(suppress)) return ER_CONTINUE; // fix up params // .. translate description = ah_translate(description); // .. caller supports a suppress flag; set the corresponding flag so that // the error display implementation enables the Suppress option. if(suppress) flags |= DE_ALLOW_SUPPRESS; if(flags & DE_NO_DEBUG_INFO) { // in non-debug-info mode, simply display the given description // and then return immediately ErrorReactionInternal er = CallDisplayError(description, flags); return PerformErrorReaction(er, flags, suppress); } // .. deal with incomplete file/line info if(!pathname || pathname[0] == '\0') pathname = L"unknown"; if(line <= 0) line = 0; if(!func || func[0] == '\0') func = "?"; // .. _FILE__ evaluates to the full path (albeit without drive letter) // which is rather long. we only display the base name for clarity. const wchar_t* filename = path_name_only(pathname); // display in output window; double-click will navigate to error location. debug_printf(L"%ls(%d): %ls\n", filename, line, description); ErrorMessageMem emm; const wchar_t* text = debug_BuildErrorMessage(description, filename, line, func, context, lastFuncToSkip, &emm); (void)debug_WriteCrashlog(text); ErrorReactionInternal er = CallDisplayError(text, flags); // note: debug_break-ing here to make sure the app doesn't continue // running is no longer necessary. debug_DisplayError now determines our // window handle and is modal. // must happen before PerformErrorReaction because that may exit. debug_FreeErrorMessage(&emm); return PerformErrorReaction(er, flags, suppress); } // is errorToSkip valid? (also guarantees mutual exclusion) enum SkipStatus { INVALID, VALID, BUSY }; static intptr_t skipStatus = INVALID; static Status errorToSkip; static size_t numSkipped; void debug_SkipErrors(Status err) { if(cpu_CAS(&skipStatus, INVALID, BUSY)) { errorToSkip = err; numSkipped = 0; COMPILER_FENCE; skipStatus = VALID; // linearization point } else DEBUG_WARN_ERR(ERR::REENTERED); } size_t debug_StopSkippingErrors() { if(cpu_CAS(&skipStatus, VALID, BUSY)) { const size_t ret = numSkipped; COMPILER_FENCE; skipStatus = INVALID; // linearization point return ret; } else { DEBUG_WARN_ERR(ERR::REENTERED); return 0; } } static bool ShouldSkipError(Status err) { if(cpu_CAS(&skipStatus, VALID, BUSY)) { numSkipped++; const bool ret = (err == errorToSkip); COMPILER_FENCE; skipStatus = VALID; return ret; } return false; } ErrorReaction debug_OnError(Status err, atomic_bool* suppress, const wchar_t* file, int line, const char* func) { if(ShouldSkipError(err)) return ER_CONTINUE; void* context = 0; const wchar_t* lastFuncToSkip = L"debug_OnError"; wchar_t buf[400]; wchar_t err_buf[200]; StatusDescription(err, err_buf, ARRAY_SIZE(err_buf)); swprintf_s(buf, ARRAY_SIZE(buf), L"Function call failed: return value was %lld (%ls)", err, err_buf); return debug_DisplayError(buf, DE_MANUAL_BREAK, context, lastFuncToSkip, file,line,func, suppress); } ErrorReaction debug_OnAssertionFailure(const wchar_t* expr, atomic_bool* suppress, const wchar_t* file, int line, const char* func) { void* context = 0; const std::wstring lastFuncToSkip = L"debug_OnAssertionFailure"; wchar_t buf[400]; swprintf_s(buf, ARRAY_SIZE(buf), L"Assertion failed: \"%ls\"", expr); return debug_DisplayError(buf, DE_MANUAL_BREAK, context, lastFuncToSkip.c_str(), file,line,func, suppress); } Index: ps/trunk/source/lib/sysdep/os/win/wposix/waio.cpp =================================================================== --- ps/trunk/source/lib/sysdep/os/win/wposix/waio.cpp (revision 9544) +++ ps/trunk/source/lib/sysdep/os/win/wposix/waio.cpp (revision 9545) @@ -1,657 +1,658 @@ /* Copyright (c) 2011 Wildfire Games * * Permission is hereby granted, free of charge, to any person obtaining * a copy of this software and associated documentation files (the * "Software"), to deal in the Software without restriction, including * without limitation the rights to use, copy, modify, merge, publish, * distribute, sublicense, and/or sell copies of the Software, and to * permit persons to whom the Software is furnished to do so, subject to * the following conditions: * * The above copyright notice and this permission notice shall be included * in all copies or substantial portions of the Software. * * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. * IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY * CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, * TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE * SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. */ /* * emulate POSIX asynchronous I/O on Windows. */ // NB: this module is significantly faster than Intel's aio library, // which also returns ERROR_INVALID_PARAMETER from aio_error if the // file is opened with FILE_FLAG_OVERLAPPED. (it looks like they are // using threaded blocking IO) #include "precompiled.h" #include "lib/sysdep/os/win/wposix/waio.h" #include "lib/bits.h" // round_up #include "lib/alignment.h" // IsAligned #include "lib/module_init.h" #include "lib/sysdep/cpu.h" // cpu_AtomicAdd #include "lib/sysdep/filesystem.h" // O_NO_AIO_NP #include "lib/sysdep/os/win/wutil.h" // wutil_SetPrivilege #include "lib/sysdep/os/win/wiocp.h" #include "lib/sysdep/os/win/winit.h" WINIT_REGISTER_MAIN_SHUTDOWN(waio_Shutdown); // (dynamic linking preserves compatibility with previous Windows versions) static WUTIL_FUNC(pSetFileCompletionNotificationModes, BOOL, (HANDLE, UCHAR)); static WUTIL_FUNC(pSetFileIoOverlappedRange, BOOL, (HANDLE, PUCHAR, ULONG)); static WUTIL_FUNC(pSetFileValidData, BOOL, (HANDLE, LONGLONG)); // (there must be one global IOCP because aio_suspend might be called for // requests from different files) static HANDLE hIOCP; //----------------------------------------------------------------------------- // OvlAllocator // allocator for OVERLAPPED (enables a special optimization, see Associate) struct OvlAllocator // POD { // freelist entries for (padded) OVERLAPPED from our storage struct Entry { SLIST_ENTRY entry; OVERLAPPED ovl; }; Status Init() { // the allocation must be naturally aligned to ensure it doesn't // overlap another page, which might prevent SetFileIoOverlappedRange // from pinning the pages if one of them is PAGE_NOACCESS. storage = _mm_malloc(storageSize, storageSize); if(!storage) WARN_RETURN(ERR::NO_MEM); memset(storage, 0, storageSize); InitializeSListHead(&freelist); // storageSize provides more than enough OVERLAPPED, so we // pad them to the cache line size to maybe avoid a few RFOs. const size_t size = Align(sizeof(Entry)); for(uintptr_t offset = 0; offset+size <= storageSize; offset += size) { Entry* entry = (Entry*)(uintptr_t(storage) + offset); ENSURE(IsAligned(entry, MEMORY_ALLOCATION_ALIGNMENT)); InterlockedPushEntrySList(&freelist, &entry->entry); } extant = 0; return INFO::OK; } void Shutdown() { - ENSURE(extant == 0); + if(extant != 0) + debug_printf(L"waio: OvlAllocator::Shutdown with extant=%d\n", extant); InterlockedFlushSList(&freelist); _mm_free(storage); storage = 0; } // irrevocably enable a special optimization for all I/Os requests // concerning this file, ending when the file is closed. has no effect // unless Vista+ and SeLockMemoryPrivilege are available. void Associate(HANDLE hFile) { ENSURE(extant == 0); // pin the page in kernel address space, which means our thread // won't have to be the one to service the I/O, thus avoiding an APC. // ("thread agnostic I/O") if(pSetFileIoOverlappedRange) WARN_IF_FALSE(pSetFileIoOverlappedRange(hFile, (PUCHAR)storage, storageSize)); } // @return OVERLAPPED initialized for I/O starting at offset, // or 0 if all available structures have already been allocated. OVERLAPPED* Allocate(off_t offset) { Entry* entry = (Entry*)InterlockedPopEntrySList(&freelist); if(!entry) return 0; OVERLAPPED& ovl = entry->ovl; ovl.Internal = 0; ovl.InternalHigh = 0; ovl.Offset = u64_lo(offset); ovl.OffsetHigh = u64_hi(offset); ovl.hEvent = 0; // (notification is via IOCP and/or polling) cpu_AtomicAdd(&extant, +1); return &ovl; } void Deallocate(OVERLAPPED* ovl) { cpu_AtomicAdd(&extant, -1); const uintptr_t address = uintptr_t(ovl); ENSURE(uintptr_t(storage) <= address && address < uintptr_t(storage)+storageSize); InterlockedPushEntrySList(&freelist, (PSLIST_ENTRY)(address - offsetof(Entry, ovl))); } // one 4 KiB page is enough for 64 OVERLAPPED per file (i.e. plenty). static const size_t storageSize = pageSize; void* storage; #if MSC_VERSION # pragma warning(push) # pragma warning(disable:4324) // structure was padded due to __declspec(align()) #endif __declspec(align(MEMORY_ALLOCATION_ALIGNMENT)) SLIST_HEADER freelist; #if MSC_VERSION # pragma warning(pop) #endif volatile intptr_t extant; }; //----------------------------------------------------------------------------- // FileControlBlock // (must correspond to static zero-initialization of fd) static const intptr_t FD_AVAILABLE = 0; // information required to start asynchronous I/Os from a file // (aiocb stores a pointer to the originating FCB) struct FileControlBlock // POD { // search key, indicates the file descriptor with which this // FCB was associated (or FD_AVAILABLE if none). volatile intptr_t fd; // second aio-enabled handle from waio_reopen HANDLE hFile; OvlAllocator ovl; Status Init() { fd = FD_AVAILABLE; hFile = INVALID_HANDLE_VALUE; return ovl.Init(); } void Shutdown() { ENSURE(fd == FD_AVAILABLE); ENSURE(hFile == INVALID_HANDLE_VALUE); ovl.Shutdown(); } }; // NB: the Windows lowio file descriptor limit is 2048, but // our applications rarely open more than a few files at a time. static FileControlBlock fileControlBlocks[16]; static FileControlBlock* AssociateFileControlBlock(int fd, HANDLE hFile) { for(size_t i = 0; i < ARRAY_SIZE(fileControlBlocks); i++) { FileControlBlock& fcb = fileControlBlocks[i]; if(cpu_CAS(&fcb.fd, FD_AVAILABLE, fd)) // the FCB is now ours { fcb.hFile = hFile; fcb.ovl.Associate(hFile); AttachToCompletionPort(hFile, hIOCP, (ULONG_PTR)&fcb); // minor optimization: avoid posting to IOCP in rare cases // where the I/O completes synchronously if(pSetFileCompletionNotificationModes) { // (for reasons as yet unknown, this fails when the file is // opened for read-only access) (void)pSetFileCompletionNotificationModes(fcb.hFile, FILE_SKIP_COMPLETION_PORT_ON_SUCCESS); } return &fcb; } } return 0; } static void DissociateFileControlBlock(FileControlBlock* fcb) { fcb->hFile = INVALID_HANDLE_VALUE; fcb->fd = FD_AVAILABLE; } static FileControlBlock* FindFileControlBlock(int fd) { ENSURE(fd != FD_AVAILABLE); for(size_t i = 0; i < ARRAY_SIZE(fileControlBlocks); i++) { FileControlBlock& fcb = fileControlBlocks[i]; if(fcb.fd == fd) return &fcb; } return 0; } //----------------------------------------------------------------------------- // init/shutdown static ModuleInitState waio_initState; static Status waio_Init() { for(size_t i = 0; i < ARRAY_SIZE(fileControlBlocks); i++) fileControlBlocks[i].Init(); WUTIL_IMPORT_KERNEL32(SetFileCompletionNotificationModes, pSetFileCompletionNotificationModes); // NB: using these functions when the privileges are not available would // trigger warnings. since callers have to check the function pointers // anyway, just refrain from setting them in such cases. if(wutil_SetPrivilege(L"SeLockMemoryPrivilege", true) == INFO::OK) WUTIL_IMPORT_KERNEL32(SetFileIoOverlappedRange, pSetFileIoOverlappedRange); if(wutil_SetPrivilege(L"SeManageVolumePrivilege", true) == INFO::OK) WUTIL_IMPORT_KERNEL32(SetFileValidData, pSetFileValidData); return INFO::OK; } static Status waio_Shutdown() { if(waio_initState == 0) // we were never initialized return INFO::OK; for(size_t i = 0; i < ARRAY_SIZE(fileControlBlocks); i++) fileControlBlocks[i].Shutdown(); WARN_IF_FALSE(CloseHandle(hIOCP)); return INFO::OK; } //----------------------------------------------------------------------------- // OpenFile static DWORD DesiredAccess(int oflag) { switch(oflag & (O_RDONLY|O_WRONLY|O_RDWR)) { case O_RDONLY: // (WinXP x64 requires FILE_WRITE_ATTRIBUTES for SetFileCompletionNotificationModes) return GENERIC_READ|FILE_WRITE_ATTRIBUTES; case O_WRONLY: return GENERIC_WRITE; case O_RDWR: return GENERIC_READ|GENERIC_WRITE; default: DEBUG_WARN_ERR(ERR::INVALID_PARAM); return 0; } } static DWORD ShareMode(int oflag) { switch(oflag & (O_RDONLY|O_WRONLY|O_RDWR)) { case O_RDONLY: return FILE_SHARE_READ; case O_WRONLY: return FILE_SHARE_WRITE; case O_RDWR: return FILE_SHARE_READ|FILE_SHARE_WRITE; default: DEBUG_WARN_ERR(ERR::INVALID_PARAM); return 0; } } static DWORD CreationDisposition(int oflag) { if(oflag & O_CREAT) return (oflag & O_EXCL)? CREATE_NEW : CREATE_ALWAYS; if(oflag & O_TRUNC) return TRUNCATE_EXISTING; return OPEN_EXISTING; } static DWORD FlagsAndAttributes() { // - FILE_FLAG_SEQUENTIAL_SCAN is ignored when FILE_FLAG_NO_BUFFERING // is set (c.f. "Windows via C/C++", p. 324) // - FILE_FLAG_WRITE_THROUGH is ~5% slower (diskspd.cpp suggests it // disables hardware caching; the overhead may also be due to the // Windows cache manager) const DWORD flags = FILE_FLAG_OVERLAPPED|FILE_FLAG_NO_BUFFERING; const DWORD attributes = FILE_ATTRIBUTE_NORMAL; return flags|attributes; } static Status OpenFile(const OsPath& pathname, int oflag, HANDLE& hFile) { WinScopedPreserveLastError s; const DWORD access = DesiredAccess(oflag); const DWORD share = ShareMode(oflag); const DWORD create = CreationDisposition(oflag); const DWORD flags = FlagsAndAttributes(); hFile = CreateFileW(OsString(pathname).c_str(), access, share, 0, create, flags, 0); if(hFile == INVALID_HANDLE_VALUE) WARN_RETURN(StatusFromWin()); return INFO::OK; } //----------------------------------------------------------------------------- // Windows-only APIs Status waio_reopen(int fd, const OsPath& pathname, int oflag, ...) { ENSURE(fd > 2); ENSURE(!(oflag & O_APPEND)); // not supported if(oflag & O_NO_AIO_NP) return INFO::SKIPPED; RETURN_STATUS_IF_ERR(ModuleInit(&waio_initState, waio_Init)); HANDLE hFile; RETURN_STATUS_IF_ERR(OpenFile(pathname, oflag, hFile)); if(!AssociateFileControlBlock(fd, hFile)) { CloseHandle(hFile); WARN_RETURN(ERR::LIMIT); } return INFO::OK; } Status waio_close(int fd) { FileControlBlock* fcb = FindFileControlBlock(fd); if(!fcb) WARN_RETURN(ERR::INVALID_HANDLE); const HANDLE hFile = fcb->hFile; DissociateFileControlBlock(fcb); if(!CloseHandle(hFile)) WARN_RETURN(ERR::INVALID_HANDLE); return INFO::OK; } Status waio_Preallocate(int fd, off_t size) { FileControlBlock* fcb = FindFileControlBlock(fd); if(!fcb) WARN_RETURN(ERR::INVALID_HANDLE); const HANDLE hFile = fcb->hFile; // Windows requires sector alignment (see discussion in header) const off_t alignedSize = round_up(size, (off_t)maxSectorSize); // (Align<> cannot compute off_t) // allocate all space up front to reduce fragmentation LARGE_INTEGER size64; size64.QuadPart = alignedSize; WARN_IF_FALSE(SetFilePointerEx(hFile, size64, 0, FILE_BEGIN)); WARN_IF_FALSE(SetEndOfFile(hFile)); // avoid synchronous zero-fill (see discussion in header) if(pSetFileValidData) { // this has been reported (#849) to fail with GetLastError() == 0 for // both tiny and medium alignedSize, despite Administrator privileges. // it seems the problem is the underlying FAT filesystem. Nick Ryan: // "FastFat does not process the FILE_VALID_DATA_LENGTH_INFORMATION IOCTL" // (http://us.generation-nt.com/answer/setfilevaliddata-help-25926952.html?page=2) // verifying the filesystem is indeed FAT would be overkill; we'll just // ignore the return code. however, GetLastError can be used to check // whether other problems arose. WinScopedPreserveLastError s; (void)pSetFileValidData(hFile, alignedSize); ENSURE(GetLastError() == 0); } return INFO::OK; } //----------------------------------------------------------------------------- // helper functions // called by aio_read, aio_write, and lio_listio. // cb->aio_lio_opcode specifies desired operation. // @return -1 on failure (having also set errno) static int Issue(aiocb* cb) { ENSURE(IsAligned(cb->aio_offset, maxSectorSize)); ENSURE(IsAligned(cb->aio_buf, maxSectorSize)); ENSURE(IsAligned(cb->aio_nbytes, maxSectorSize)); FileControlBlock* fcb = FindFileControlBlock(cb->aio_fildes); if(!fcb || fcb->hFile == INVALID_HANDLE_VALUE) { DEBUG_WARN_ERR(ERR::INVALID_HANDLE); errno = EINVAL; return -1; } ENSURE(!cb->fcb && !cb->ovl); // SUSv3: aiocb must not be in use cb->fcb = fcb; cb->ovl = fcb->ovl.Allocate(cb->aio_offset); if(!cb->ovl) { DEBUG_WARN_ERR(ERR::LIMIT); errno = EMFILE; return -1; } WinScopedPreserveLastError s; const HANDLE hFile = fcb->hFile; void* const buf = (void*)cb->aio_buf; // from volatile void* const DWORD size = u64_lo(cb->aio_nbytes); ENSURE(u64_hi(cb->aio_nbytes) == 0); OVERLAPPED* ovl = (OVERLAPPED*)cb->ovl; // (there is no point in using WriteFileGather/ReadFileScatter here // because the IO manager still needs to lock pages and translate them // into an MDL, and we'd just be increasing the number of addresses) const BOOL ok = (cb->aio_lio_opcode == LIO_WRITE)? WriteFile(hFile, buf, size, 0, ovl) : ReadFile(hFile, buf, size, 0, ovl); if(ok || GetLastError() == ERROR_IO_PENDING) return 0; // success const Status status = StatusFromWin(); WARN_IF_ERR(status); errno = ErrnoFromStatus(status); return -1; } static bool AreAnyComplete(const struct aiocb* const cbs[], int n) { for(int i = 0; i < n; i++) { if(!cbs[i]) // SUSv3: must ignore NULL entries continue; if(HasOverlappedIoCompleted((OVERLAPPED*)cbs[i]->ovl)) return true; } return false; } //----------------------------------------------------------------------------- // API int aio_read(struct aiocb* cb) { cb->aio_lio_opcode = LIO_READ; return Issue(cb); } int aio_write(struct aiocb* cb) { cb->aio_lio_opcode = LIO_WRITE; return Issue(cb); } int lio_listio(int mode, struct aiocb* const cbs[], int n, struct sigevent* se) { ENSURE(mode == LIO_WAIT || mode == LIO_NOWAIT); UNUSED2(se); // signaling is not implemented. for(int i = 0; i < n; i++) { if(cbs[i] == 0 || cbs[i]->aio_lio_opcode == LIO_NOP) continue; if(Issue(cbs[i]) == -1) return -1; } if(mode == LIO_WAIT) return aio_suspend(cbs, n, 0); return 0; } int aio_suspend(const struct aiocb* const cbs[], int n, const struct timespec* timeout) { // consume all pending notifications to prevent them from piling up if // requests are always complete by the time we're called DWORD bytesTransferred; ULONG_PTR key; OVERLAPPED* ovl; while(PollCompletionPort(hIOCP, 0, bytesTransferred, key, ovl) == INFO::OK) {} // avoid blocking if already complete (synchronous requests don't post notifications) if(AreAnyComplete(cbs, n)) return 0; // caller doesn't want to block, and no requests are complete if(timeout && timeout->tv_sec == 0 && timeout->tv_nsec == 0) { errno = EAGAIN; return -1; } // reduce CPU usage by blocking until a notification arrives or a // brief timeout elapses (necessary because other threads - or even // the above poll - might have consumed our notification). note that // re-posting notifications that don't concern the respective requests // is not desirable because POSIX doesn't require aio_suspend to be // called, which means notifications might pile up. const DWORD milliseconds = 1; // as short as possible (don't oversleep) const Status ret = PollCompletionPort(hIOCP, milliseconds, bytesTransferred, key, ovl); if(ret != INFO::OK && ret != ERR::AGAIN) // failed { DEBUG_WARN_ERR(ERR::LOGIC); return -1; } // scan again (even if we got a notification, it might not concern THESE requests) if(AreAnyComplete(cbs, n)) return 0; // none completed, must repeat the above steps. provoke being called again by // claiming to have been interrupted by a signal. errno = EINTR; return -1; } int aio_error(const struct aiocb* cb) { const OVERLAPPED* ovl = (const OVERLAPPED*)cb->ovl; if(!ovl) // called after aio_return return EINVAL; if(!HasOverlappedIoCompleted(ovl)) return EINPROGRESS; if(ovl->Internal != ERROR_SUCCESS) return EIO; return 0; } ssize_t aio_return(struct aiocb* cb) { FileControlBlock* fcb = (FileControlBlock*)cb->fcb; OVERLAPPED* ovl = (OVERLAPPED*)cb->ovl; if(!fcb || !ovl) { errno = EINVAL; return -1; } const ULONG_PTR status = ovl->Internal; const ULONG_PTR bytesTransferred = ovl->InternalHigh; cb->ovl = 0; // prevent further calls to aio_error/aio_return COMPILER_FENCE; fcb->ovl.Deallocate(ovl); cb->fcb = 0; // allow reuse return (status == ERROR_SUCCESS)? bytesTransferred : -1; } int aio_cancel(int UNUSED(fd), struct aiocb* cb) { // (faster than calling FindFileControlBlock) const HANDLE hFile = ((const FileControlBlock*)cb->fcb)->hFile; if(hFile == INVALID_HANDLE_VALUE) { WARN_IF_ERR(ERR::INVALID_HANDLE); errno = EINVAL; return -1; } // cancel all I/Os this thread issued for the given file // (CancelIoEx can cancel individual operations, but is only // available starting with Vista) WARN_IF_FALSE(CancelIo(hFile)); return AIO_CANCELED; } int aio_fsync(int, struct aiocb*) { WARN_IF_ERR(ERR::NOT_SUPPORTED); errno = ENOSYS; return -1; } Index: ps/trunk/source/lib/sysdep/os/win/wiocp.cpp =================================================================== --- ps/trunk/source/lib/sysdep/os/win/wiocp.cpp (revision 9544) +++ ps/trunk/source/lib/sysdep/os/win/wiocp.cpp (revision 9545) @@ -1,34 +1,34 @@ #include "precompiled.h" #include "lib/sysdep/os/win/wiocp.h" #include "lib/file/file.h" // ERR::IO #include "lib/sysdep/os/win/wutil.h" void AttachToCompletionPort(HANDLE hFile, HANDLE& hIOCP, ULONG_PTR key, DWORD numConcurrentThreads) { WinScopedPreserveLastError s; // CreateIoCompletionPort // (when called for the first time, ends up creating hIOCP) hIOCP = CreateIoCompletionPort(hFile, hIOCP, key, numConcurrentThreads); ENSURE(wutil_IsValidHandle(hIOCP)); } Status PollCompletionPort(HANDLE hIOCP, DWORD timeout, DWORD& bytesTransferred, ULONG_PTR& key, OVERLAPPED*& ovl) { if(hIOCP == 0) return ERR::INVALID_HANDLE; // NOWARN (happens if called before the first Attach) WinScopedPreserveLastError s; bytesTransferred = 0; key = 0; ovl = 0; if(GetQueuedCompletionStatus(hIOCP, &bytesTransferred, &key, &ovl, timeout)) return INFO::OK; const Status ret = StatusFromWin(); if(ret == ERR::AGAIN || ret == ERR::ABORTED) // avoid polluting last error SetLastError(0); - return StatusFromWin(); // NOWARN (let caller decide what to do) + return ret; // NOWARN (let caller decide what to do) } Index: ps/trunk/source/lib/sysdep/os/win/wsysdep.cpp =================================================================== --- ps/trunk/source/lib/sysdep/os/win/wsysdep.cpp (revision 9544) +++ ps/trunk/source/lib/sysdep/os/win/wsysdep.cpp (revision 9545) @@ -1,608 +1,612 @@ /* Copyright (c) 2011 Wildfire Games * * Permission is hereby granted, free of charge, to any person obtaining * a copy of this software and associated documentation files (the * "Software"), to deal in the Software without restriction, including * without limitation the rights to use, copy, modify, merge, publish, * distribute, sublicense, and/or sell copies of the Software, and to * permit persons to whom the Software is furnished to do so, subject to * the following conditions: * * The above copyright notice and this permission notice shall be included * in all copies or substantial portions of the Software. * * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. * IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY * CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, * TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE * SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. */ /* * Windows backend of the sysdep interface */ #include "precompiled.h" #include "lib/sysdep/sysdep.h" #include "lib/alignment.h" #include "lib/sysdep/os/win/win.h" // includes windows.h; must come before shlobj #include // pick_dir #include // open_url #include #include // message crackers #include #include "lib/sysdep/clipboard.h" #include "lib/sysdep/os/win/error_dialog.h" #include "lib/sysdep/os/win/wutil.h" #if CONFIG_ENABLE_BOOST # include #endif #if MSC_VERSION #pragma comment(lib, "shell32.lib") // for sys_pick_directory SH* calls #pragma comment(lib, "winhttp.lib") #endif bool sys_IsDebuggerPresent() { return (IsDebuggerPresent() != 0); } std::wstring sys_WideFromArgv(const char* argv_i) { // NB: despite http://cbloomrants.blogspot.com/2008/06/06-14-08-1.html, // WinXP x64 EN cmd.exe (chcp reports 437) encodes argv u-umlaut // (entered manually or via auto-complete) via cp1252. the same applies // to WinXP SP2 DE (where chcp reports 850). const UINT cp = CP_ACP; const DWORD flags = MB_PRECOMPOSED|MB_ERR_INVALID_CHARS; const int inputSize = -1; // null-terminated std::vector buf(strlen(argv_i)+1); // (upper bound on number of characters) // NB: avoid mbstowcs because it may specify another locale const int ret = MultiByteToWideChar(cp, flags, argv_i, (int)inputSize, &buf[0], (int)buf.size()); ENSURE(ret != 0); return std::wstring(&buf[0]); } void sys_display_msg(const wchar_t* caption, const wchar_t* msg) { MessageBoxW(0, msg, caption, MB_ICONEXCLAMATION|MB_TASKMODAL|MB_SETFOREGROUND); } //----------------------------------------------------------------------------- // "program error" dialog (triggered by ENSURE and exception) //----------------------------------------------------------------------------- // support for resizing the dialog / its controls (must be done manually) static POINTS dlg_clientOrigin; static POINTS dlg_prevClientSize; static void dlg_OnMove(HWND UNUSED(hDlg), int x, int y) { dlg_clientOrigin.x = (short)x; dlg_clientOrigin.y = (short)y; } static const size_t ANCHOR_LEFT = 0x01; static const size_t ANCHOR_RIGHT = 0x02; static const size_t ANCHOR_TOP = 0x04; static const size_t ANCHOR_BOTTOM = 0x08; static const size_t ANCHOR_ALL = 0x0F; static void dlg_ResizeControl(HWND hDlg, int dlgItem, int dx, int dy, size_t anchors) { HWND hControl = GetDlgItem(hDlg, dlgItem); RECT r; GetWindowRect(hControl, &r); int w = r.right - r.left, h = r.bottom - r.top; int x = r.left - dlg_clientOrigin.x, y = r.top - dlg_clientOrigin.y; if(anchors & ANCHOR_RIGHT) { // right only if(!(anchors & ANCHOR_LEFT)) x += dx; // horizontal (stretch width) else w += dx; } if(anchors & ANCHOR_BOTTOM) { // bottom only if(!(anchors & ANCHOR_TOP)) y += dy; // vertical (stretch height) else h += dy; } SetWindowPos(hControl, 0, x,y, w,h, SWP_NOZORDER); } static void dlg_OnSize(HWND hDlg, UINT state, int clientSizeX, int clientSizeY) { // 'minimize' was clicked. we need to ignore this, otherwise // dx/dy would reduce some control positions to less than 0. // since Windows clips them, we wouldn't later be able to // reconstruct the previous values when 'restoring'. if(state == SIZE_MINIMIZED) return; // NB: origin might legitimately be 0, but we know it is invalid // on the first call to this function, where dlg_prevClientSize is 0. const bool isOriginValid = (dlg_prevClientSize.y != 0); const int dx = clientSizeX - dlg_prevClientSize.x; const int dy = clientSizeY - dlg_prevClientSize.y; dlg_prevClientSize.x = (short)clientSizeX; dlg_prevClientSize.y = (short)clientSizeY; if(!isOriginValid) // must not call dlg_ResizeControl return; dlg_ResizeControl(hDlg, IDC_CONTINUE, dx,dy, ANCHOR_LEFT|ANCHOR_BOTTOM); dlg_ResizeControl(hDlg, IDC_SUPPRESS, dx,dy, ANCHOR_LEFT|ANCHOR_BOTTOM); dlg_ResizeControl(hDlg, IDC_BREAK , dx,dy, ANCHOR_LEFT|ANCHOR_BOTTOM); dlg_ResizeControl(hDlg, IDC_EXIT , dx,dy, ANCHOR_LEFT|ANCHOR_BOTTOM); dlg_ResizeControl(hDlg, IDC_COPY , dx,dy, ANCHOR_RIGHT|ANCHOR_BOTTOM); dlg_ResizeControl(hDlg, IDC_EDIT1 , dx,dy, ANCHOR_ALL); } static void dlg_OnGetMinMaxInfo(HWND UNUSED(hDlg), LPMINMAXINFO mmi) { // we must make sure resize_control will never set negative coords - // Windows would clip them, and its real position would be lost. // restrict to a reasonable and good looking minimum size [pixels]. mmi->ptMinTrackSize.x = 407; mmi->ptMinTrackSize.y = 159; // determined experimentally } struct DialogParams { const wchar_t* text; size_t flags; }; static BOOL dlg_OnInitDialog(HWND hDlg, HWND UNUSED(hWndFocus), LPARAM lParam) { const DialogParams* params = (const DialogParams*)lParam; HWND hWnd; // need to reset for new instance of dialog dlg_clientOrigin.x = dlg_clientOrigin.y = 0; dlg_prevClientSize.x = dlg_prevClientSize.y = 0; if(!(params->flags & DE_ALLOW_SUPPRESS)) { hWnd = GetDlgItem(hDlg, IDC_SUPPRESS); EnableWindow(hWnd, FALSE); } // set fixed font for readability hWnd = GetDlgItem(hDlg, IDC_EDIT1); HGDIOBJ hObj = (HGDIOBJ)GetStockObject(SYSTEM_FIXED_FONT); LPARAM redraw = FALSE; SendMessage(hWnd, WM_SETFONT, (WPARAM)hObj, redraw); SetDlgItemTextW(hDlg, IDC_EDIT1, params->text); return TRUE; // set default keyboard focus } static void dlg_OnCommand(HWND hDlg, int id, HWND UNUSED(hWndCtl), UINT UNUSED(codeNotify)) { switch(id) { case IDC_COPY: { std::vector buf(128*KiB); // (too big for stack) GetDlgItemTextW(hDlg, IDC_EDIT1, &buf[0], (int)buf.size()); sys_clipboard_set(&buf[0]); break; } case IDC_CONTINUE: EndDialog(hDlg, ERI_CONTINUE); break; case IDC_SUPPRESS: EndDialog(hDlg, ERI_SUPPRESS); break; case IDC_BREAK: EndDialog(hDlg, ERI_BREAK); break; case IDC_EXIT: EndDialog(hDlg, ERI_EXIT); break; default: break; } } static void dlg_OnSysCommand(HWND hDlg, UINT cmd, int UNUSED(x), int UNUSED(y)) { switch(cmd & 0xFFF0) // NB: lower 4 bits are reserved { // [X] clicked -> close dialog (doesn't happen automatically) case SC_CLOSE: EndDialog(hDlg, 0); break; default: break; } } static INT_PTR CALLBACK dlg_OnMessage(HWND hDlg, unsigned int msg, WPARAM wParam, LPARAM lParam) { switch(msg) { case WM_INITDIALOG: return HANDLE_WM_INITDIALOG(hDlg, wParam, lParam, dlg_OnInitDialog); case WM_SYSCOMMAND: return HANDLE_WM_SYSCOMMAND(hDlg, wParam, lParam, dlg_OnSysCommand); case WM_COMMAND: return HANDLE_WM_COMMAND(hDlg, wParam, lParam, dlg_OnCommand); case WM_MOVE: return HANDLE_WM_MOVE(hDlg, wParam, lParam, dlg_OnMove); case WM_GETMINMAXINFO: return HANDLE_WM_GETMINMAXINFO(hDlg, wParam, lParam, dlg_OnGetMinMaxInfo); case WM_SIZE: return HANDLE_WM_SIZE(hDlg, wParam, lParam, dlg_OnSize); default: // we didn't process the message; caller will perform default action. return FALSE; } } ErrorReactionInternal sys_display_error(const wchar_t* text, size_t flags) { // note: other threads might still be running, crash and take down the // process before we have a chance to display this error message. // ideally we would suspend them all and resume when finished; however, // they may be holding system-wide locks (e.g. heap or loader) that // are potentially needed by DialogBoxParam. in that case, deadlock // would result; this is much worse than a crash because no error // at all is displayed to the end-user. therefore, do nothing here. // temporarily remove any pending quit message from the queue because // it would prevent the dialog from being displayed (DialogBoxParam // returns IDOK without doing anything). will be restored below. // notes: // - this isn't only relevant at exit - Windows also posts one if // window init fails. therefore, it is important that errors can be // displayed regardless. // - by passing hWnd=0, we check all windows belonging to the current // thread. there is no reason to use hWndParent below. MSG msg; const BOOL isQuitPending = PeekMessage(&msg, 0, WM_QUIT, WM_QUIT, PM_REMOVE); const HINSTANCE hInstance = wutil_LibModuleHandle(); LPCWSTR lpTemplateName = MAKEINTRESOURCEW(IDD_DIALOG1); const DialogParams params = { text, flags }; // get the enclosing app's window handle. we can't just pass 0 or // the desktop window because the dialog must be modal (if the app // continues running, it may crash and take down the process before // we've managed to show the dialog). const HWND hWndParent = wutil_AppWindow(); INT_PTR ret = DialogBoxParamW(hInstance, lpTemplateName, hWndParent, dlg_OnMessage, (LPARAM)¶ms); if(isQuitPending) PostQuitMessage((int)msg.wParam); // failed; warn user and make sure we return an ErrorReactionInternal. if(ret == 0 || ret == -1) { debug_DisplayMessage(L"Error", L"Unable to display detailed error dialog."); return ERI_CONTINUE; } return (ErrorReactionInternal)ret; } //----------------------------------------------------------------------------- // misc //----------------------------------------------------------------------------- Status sys_StatusDescription(int user_err, wchar_t* buf, size_t max_chars) { // validate user_err - Win32 doesn't have negative error numbers if(user_err < 0) return ERR::FAIL; // NOWARN const DWORD err = user_err? (DWORD)user_err : GetLastError(); // no one likes to see "The operation completed successfully" in // error messages, so return more descriptive text instead. if(err == 0) { wcscpy_s(buf, max_chars, L"0 (no error code was set)"); return INFO::OK; } wchar_t message[200]; { const LPCVOID source = 0; // ignored (we're not using FROM_HMODULE etc.) const DWORD lang_id = 0; // look for neutral, then current locale va_list* args = 0; // we don't care about "inserts" const DWORD charsWritten = FormatMessageW(FORMAT_MESSAGE_FROM_SYSTEM, source, err, lang_id, message, (DWORD)ARRAY_SIZE(message), args); if(!charsWritten) WARN_RETURN(ERR::FAIL); ENSURE(charsWritten < max_chars); + if(message[charsWritten-1] == '\n') + message[charsWritten-1] = '\0'; + if(message[charsWritten-2] == '\r') + message[charsWritten-2] = '\0'; } const int charsWritten = swprintf_s(buf, max_chars, L"%d (%ls)", err, message); ENSURE(charsWritten != -1); return INFO::OK; } static Status GetModulePathname(HMODULE hModule, OsPath& pathname) { wchar_t pathnameBuf[32768]; // NTFS limit const DWORD length = (DWORD)ARRAY_SIZE(pathnameBuf); const DWORD charsWritten = GetModuleFileNameW(hModule, pathnameBuf, length); if(charsWritten == 0) // failed WARN_RETURN(StatusFromWin()); ENSURE(charsWritten < length); // why would the above buffer ever be exceeded? pathname = pathnameBuf; return INFO::OK; } Status sys_get_module_filename(void* addr, OsPath& pathname) { MEMORY_BASIC_INFORMATION mbi; const SIZE_T bytesWritten = VirtualQuery(addr, &mbi, sizeof(mbi)); if(!bytesWritten) WARN_RETURN(StatusFromWin()); ENSURE(bytesWritten >= sizeof(mbi)); return GetModulePathname((HMODULE)mbi.AllocationBase, pathname); } OsPath sys_ExecutablePathname() { WinScopedPreserveLastError s; OsPath pathname; ENSURE(GetModulePathname(0, pathname) == INFO::OK); return pathname; } std::wstring sys_get_user_name() { wchar_t usernameBuf[256]; DWORD size = ARRAY_SIZE(usernameBuf); if(!GetUserNameW(usernameBuf, &size)) return L""; return usernameBuf; } // callback for shell directory picker: used to set starting directory // (for user convenience). static int CALLBACK BrowseCallback(HWND hWnd, unsigned int msg, LPARAM UNUSED(lParam), LPARAM lpData) { if(msg == BFFM_INITIALIZED) { const WPARAM wParam = TRUE; // lpData is a Unicode string, not PIDL. // (MSDN: the return values for both of these BFFM_ notifications are ignored) (void)SendMessage(hWnd, BFFM_SETSELECTIONW, wParam, lpData); } return 0; } Status sys_pick_directory(OsPath& path) { // (must not use multi-threaded apartment due to BIF_NEWDIALOGSTYLE) const HRESULT hr = CoInitialize(0); ENSURE(hr == S_OK || hr == S_FALSE); // S_FALSE == already initialized // note: bi.pszDisplayName isn't the full path, so it isn't of any use. BROWSEINFOW bi; memset(&bi, 0, sizeof(bi)); bi.ulFlags = BIF_RETURNONLYFSDIRS|BIF_NEWDIALOGSTYLE|BIF_NONEWFOLDERBUTTON; // for setting starting directory: bi.lpfn = (BFFCALLBACK)BrowseCallback; const Path::String initialPath = OsString(path); // NB: BFFM_SETSELECTIONW can't deal with '/' separators bi.lParam = (LPARAM)initialPath.c_str(); const LPITEMIDLIST pidl = SHBrowseForFolderW(&bi); if(!pidl) // user canceled return INFO::SKIPPED; // translate ITEMIDLIST to string wchar_t pathBuf[MAX_PATH]; // mandated by SHGetPathFromIDListW const BOOL ok = SHGetPathFromIDListW(pidl, pathBuf); // free the ITEMIDLIST IMalloc* p_malloc; SHGetMalloc(&p_malloc); p_malloc->Free(pidl); p_malloc->Release(); if(ok == TRUE) { path = pathBuf; return INFO::OK; } WARN_RETURN(StatusFromWin()); } Status sys_open_url(const std::string& url) { HINSTANCE r = ShellExecuteA(NULL, "open", url.c_str(), NULL, NULL, SW_SHOWNORMAL); if ((int)(intptr_t)r > 32) return INFO::OK; WARN_RETURN(ERR::FAIL); } Status sys_generate_random_bytes(u8* buffer, size_t size) { HCRYPTPROV hCryptProv = 0; if(!CryptAcquireContext(&hCryptProv, 0, 0, PROV_RSA_FULL, CRYPT_VERIFYCONTEXT)) WARN_RETURN(StatusFromWin()); memset(buffer, 0, size); if(!CryptGenRandom(hCryptProv, (DWORD)size, (BYTE*)buffer)) WARN_RETURN(StatusFromWin()); if(!CryptReleaseContext(hCryptProv, 0)) WARN_RETURN(StatusFromWin()); return INFO::OK; } #if CONFIG_ENABLE_BOOST /* * Given a string of the form * "example.com:80" * or * "ftp=ftp.example.com:80;http=example.com:80;https=example.com:80" * separated by semicolons or whitespace, * return the string "example.com:80". */ static std::wstring parse_proxy(const std::wstring& input) { if(input.find('=') == input.npos) return input; std::vector parts; split(parts, input, boost::algorithm::is_any_of("; \t\r\n"), boost::algorithm::token_compress_on); for(size_t i = 0; i < parts.size(); ++i) if(boost::algorithm::starts_with(parts[i], "http=")) return parts[i].substr(5); // If we got this far, proxies were only set for non-HTTP protocols return L""; } Status sys_get_proxy_config(const std::wstring& url, std::wstring& proxy) { WINHTTP_AUTOPROXY_OPTIONS autoProxyOptions; memset(&autoProxyOptions, 0, sizeof(autoProxyOptions)); autoProxyOptions.dwFlags = WINHTTP_AUTOPROXY_AUTO_DETECT; autoProxyOptions.dwAutoDetectFlags = WINHTTP_AUTO_DETECT_TYPE_DHCP | WINHTTP_AUTO_DETECT_TYPE_DNS_A; autoProxyOptions.fAutoLogonIfChallenged = TRUE; WINHTTP_PROXY_INFO proxyInfo; memset(&proxyInfo, 0, sizeof(proxyInfo)); WINHTTP_CURRENT_USER_IE_PROXY_CONFIG ieConfig; memset(&ieConfig, 0, sizeof(ieConfig)); HINTERNET hSession = NULL; Status err = INFO::SKIPPED; bool useAutoDetect; if(WinHttpGetIEProxyConfigForCurrentUser(&ieConfig)) { if(ieConfig.lpszAutoConfigUrl) { // Use explicit auto-config script if specified useAutoDetect = true; autoProxyOptions.dwFlags |= WINHTTP_AUTOPROXY_CONFIG_URL; autoProxyOptions.lpszAutoConfigUrl = ieConfig.lpszAutoConfigUrl; } else { // Use auto-discovery if enabled useAutoDetect = (ieConfig.fAutoDetect == TRUE); } } else { // Can't find IE config settings - fall back to auto-discovery useAutoDetect = true; } if(useAutoDetect) { hSession = WinHttpOpen(NULL, WINHTTP_ACCESS_TYPE_DEFAULT_PROXY, WINHTTP_NO_PROXY_NAME, WINHTTP_NO_PROXY_BYPASS, 0); if(hSession && WinHttpGetProxyForUrl(hSession, url.c_str(), &autoProxyOptions, &proxyInfo) && proxyInfo.lpszProxy) { proxy = parse_proxy(proxyInfo.lpszProxy); if(!proxy.empty()) { err = INFO::OK; goto done; } } } // No valid auto-config; try explicit proxy instead if(ieConfig.lpszProxy) { proxy = parse_proxy(ieConfig.lpszProxy); if(!proxy.empty()) { err = INFO::OK; goto done; } } done: if(ieConfig.lpszProxy) GlobalFree(ieConfig.lpszProxy); if(ieConfig.lpszProxyBypass) GlobalFree(ieConfig.lpszProxyBypass); if(ieConfig.lpszAutoConfigUrl) GlobalFree(ieConfig.lpszAutoConfigUrl); if(proxyInfo.lpszProxy) GlobalFree(proxyInfo.lpszProxy); if(proxyInfo.lpszProxyBypass) GlobalFree(proxyInfo.lpszProxyBypass); if(hSession) WinHttpCloseHandle(hSession); return err; } #endif FILE* sys_OpenFile(const OsPath& pathname, const char* mode) { FILE* f = 0; const std::wstring wmode(mode, mode+strlen(mode)); (void)_wfopen_s(&f, OsString(pathname).c_str(), wmode.c_str()); return f; } Index: ps/trunk/source/lib/sysdep/os/win/wnuma.cpp =================================================================== --- ps/trunk/source/lib/sysdep/os/win/wnuma.cpp (revision 9544) +++ ps/trunk/source/lib/sysdep/os/win/wnuma.cpp (revision 9545) @@ -1,569 +1,570 @@ /* Copyright (c) 2010 Wildfire Games * * Permission is hereby granted, free of charge, to any person obtaining * a copy of this software and associated documentation files (the * "Software"), to deal in the Software without restriction, including * without limitation the rights to use, copy, modify, merge, publish, * distribute, sublicense, and/or sell copies of the Software, and to * permit persons to whom the Software is furnished to do so, subject to * the following conditions: * * The above copyright notice and this permission notice shall be included * in all copies or substantial portions of the Software. * * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. * IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY * CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, * TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE * SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. */ #include "precompiled.h" #include "lib/sysdep/numa.h" #include "lib/bits.h" // PopulationCount #include "lib/alignment.h" #include "lib/timer.h" #include "lib/module_init.h" #include "lib/allocators/page_aligned.h" #include "lib/sysdep/os_cpu.h" #include "lib/sysdep/acpi.h" #include "lib/sysdep/os/win/win.h" #include "lib/sysdep/os/win/wutil.h" #include "lib/sysdep/os/win/wcpu.h" #include #if ARCH_X86_X64 #include "lib/sysdep/arch/x86_x64/topology.h" // ApicIds #endif //----------------------------------------------------------------------------- // nodes struct Node // POD { // (Windows doesn't guarantee node numbers are contiguous, so // we associate them with contiguous indices in nodes[]) UCHAR nodeNumber; u32 proximityDomainNumber; uintptr_t processorMask; }; static Node nodes[os_cpu_MaxProcessors]; static size_t numNodes; static Node* AddNode() { ENSURE(numNodes < ARRAY_SIZE(nodes)); return &nodes[numNodes++]; } static Node* FindNodeWithProcessorMask(uintptr_t processorMask) { for(size_t node = 0; node < numNodes; node++) { if(nodes[node].processorMask == processorMask) return &nodes[node]; } return 0; } static Node* FindNodeWithProcessor(size_t processor) { for(size_t node = 0; node < numNodes; node++) { if(IsBitSet(nodes[node].processorMask, processor)) return &nodes[node]; } return 0; } // cached results of FindNodeWithProcessor for each processor static size_t processorsNode[os_cpu_MaxProcessors]; static void FillProcessorsNode() { for(size_t processor = 0; processor < os_cpu_NumProcessors(); processor++) { Node* node = FindNodeWithProcessor(processor); if(node) processorsNode[processor] = node-nodes; else DEBUG_WARN_ERR(ERR::LOGIC); } } //----------------------------------------------------------------------------- // Windows topology static UCHAR HighestNodeNumber() { WUTIL_FUNC(pGetNumaHighestNodeNumber, BOOL, (PULONG)); WUTIL_IMPORT_KERNEL32(GetNumaHighestNodeNumber, pGetNumaHighestNodeNumber); if(!pGetNumaHighestNodeNumber) return 0; // NUMA not supported => only one node ULONG highestNodeNumber; const BOOL ok = pGetNumaHighestNodeNumber(&highestNodeNumber); WARN_IF_FALSE(ok); return (UCHAR)highestNodeNumber; } static void PopulateNodes() { WUTIL_FUNC(pGetNumaNodeProcessorMask, BOOL, (UCHAR, PULONGLONG)); WUTIL_IMPORT_KERNEL32(GetNumaNodeProcessorMask, pGetNumaNodeProcessorMask); if(!pGetNumaNodeProcessorMask) return; DWORD_PTR processAffinity, systemAffinity; { const BOOL ok = GetProcessAffinityMask(GetCurrentProcess(), &processAffinity, &systemAffinity); WARN_IF_FALSE(ok); } ENSURE(PopulationCount(processAffinity) <= PopulationCount(systemAffinity)); for(UCHAR nodeNumber = 0; nodeNumber <= HighestNodeNumber(); nodeNumber++) { ULONGLONG affinity; { const BOOL ok = pGetNumaNodeProcessorMask(nodeNumber, &affinity); WARN_IF_FALSE(ok); } if(!affinity) continue; // empty node, skip Node* node = AddNode(); node->nodeNumber = nodeNumber; node->processorMask = wcpu_ProcessorMaskFromAffinity(processAffinity, (DWORD_PTR)affinity); } } //----------------------------------------------------------------------------- // ACPI SRAT topology #if ARCH_X86_X64 #pragma pack(push, 1) // fields common to Affinity* structures struct AffinityHeader { u8 type; u8 length; // size [bytes], including this header }; struct AffinityAPIC { static const u8 type = 0; AffinityHeader header; u8 proximityDomainNumber0; u8 apicId; u32 flags; u8 sapicId; u8 proximityDomainNumber123[3]; u32 clockDomain; u32 ProximityDomainNumber() const { // (this is the apparent result of backwards compatibility, ugh.) u32 proximityDomainNumber; memcpy(&proximityDomainNumber, &proximityDomainNumber123[0]-1, sizeof(proximityDomainNumber)); proximityDomainNumber &= ~0xFF; proximityDomainNumber |= proximityDomainNumber0; return proximityDomainNumber; } }; struct AffinityMemory { static const u8 type = 1; AffinityHeader header; u32 proximityDomainNumber; u16 reserved1; u64 baseAddress; u64 length; u32 reserved2; u32 flags; u64 reserved3; }; // AffinityX2APIC omitted, since the APIC ID is sufficient for our purposes // Static Resource Affinity Table struct SRAT { AcpiTable header; u32 reserved1; u8 reserved2[8]; AffinityHeader affinities[1]; }; #pragma pack(pop) template static const Affinity* DynamicCastFromHeader(const AffinityHeader* header) { if(header->type != Affinity::type) return 0; // sanity check: ensure no padding was inserted ENSURE(header->length == sizeof(Affinity)); const Affinity* affinity = (const Affinity*)header; if(!IsBitSet(affinity->flags, 0)) // not enabled return 0; return affinity; } static void PopulateProcessorMaskFromApicId(u32 apicId, uintptr_t& processorMask) { const u8* apicIds = ApicIds(); for(size_t processor = 0; processor < os_cpu_NumProcessors(); processor++) { if(apicIds[processor] == apicId) { processorMask |= Bit(processor); return; } } DEBUG_WARN_ERR(ERR::LOGIC); // APIC ID not found } struct ProximityDomain { uintptr_t processorMask; // (AffinityMemory's fields are not currently needed) }; typedef std::map ProximityDomains; static ProximityDomains ExtractProximityDomainsFromSRAT(const SRAT* srat) { ProximityDomains proximityDomains; for(const AffinityHeader* header = srat->affinities; header < (const AffinityHeader*)(uintptr_t(srat)+srat->header.size); header = (const AffinityHeader*)(uintptr_t(header) + header->length)) { const AffinityAPIC* affinityAPIC = DynamicCastFromHeader(header); if(affinityAPIC) { const u32 proximityDomainNumber = affinityAPIC->ProximityDomainNumber(); ProximityDomain& proximityDomain = proximityDomains[proximityDomainNumber]; PopulateProcessorMaskFromApicId(affinityAPIC->apicId, proximityDomain.processorMask); } } return proximityDomains; } static void PopulateNodesFromProximityDomains(const ProximityDomains& proximityDomains) { for(ProximityDomains::const_iterator it = proximityDomains.begin(); it != proximityDomains.end(); ++it) { const u32 proximityDomainNumber = it->first; const ProximityDomain& proximityDomain = it->second; Node* node = FindNodeWithProcessorMask(proximityDomain.processorMask); if(!node) node = AddNode(); node->proximityDomainNumber = proximityDomainNumber; node->processorMask = proximityDomain.processorMask; } } #endif // #if ARCH_X86_X64 //----------------------------------------------------------------------------- static ModuleInitState initState; static Status InitTopology() { PopulateNodes(); #if ARCH_X86_X64 const SRAT* srat = (const SRAT*)acpi_GetTable("SRAT"); if(srat) { const ProximityDomains proximityDomains = ExtractProximityDomainsFromSRAT(srat); PopulateNodesFromProximityDomains(proximityDomains); } #endif // neither OS nor ACPI information is available if(numNodes == 0) { // add dummy node that contains all system processors Node* node = AddNode(); node->nodeNumber = 0; node->proximityDomainNumber = 0; node->processorMask = os_cpu_ProcessorMask(); } FillProcessorsNode(); return INFO::OK; } size_t numa_NumNodes() { (void)ModuleInit(&initState, InitTopology); return numNodes; } size_t numa_NodeFromProcessor(size_t processor) { (void)ModuleInit(&initState, InitTopology); ENSURE(processor < os_cpu_NumProcessors()); return processorsNode[processor]; } uintptr_t numa_ProcessorMaskFromNode(size_t node) { (void)ModuleInit(&initState, InitTopology); ENSURE(node < numNodes); return nodes[node].processorMask; } static UCHAR NodeNumberFromNode(size_t node) { (void)ModuleInit(&initState, InitTopology); ENSURE(node < numa_NumNodes()); return nodes[node].nodeNumber; } //----------------------------------------------------------------------------- // memory info size_t numa_AvailableMemory(size_t node) { // note: it is said that GetNumaAvailableMemoryNode sometimes incorrectly // reports zero bytes. the actual cause may however be unexpected // RAM configuration, e.g. not all slots filled. WUTIL_FUNC(pGetNumaAvailableMemoryNode, BOOL, (UCHAR, PULONGLONG)); WUTIL_IMPORT_KERNEL32(GetNumaAvailableMemoryNode, pGetNumaAvailableMemoryNode); if(pGetNumaAvailableMemoryNode) { const UCHAR nodeNumber = NodeNumberFromNode(node); ULONGLONG availableBytes; const BOOL ok = pGetNumaAvailableMemoryNode(nodeNumber, &availableBytes); WARN_IF_FALSE(ok); const size_t availableMiB = size_t(availableBytes / MiB); return availableMiB; } // NUMA not supported - return available system memory else return os_cpu_MemoryAvailable(); } #pragma pack(push, 1) // ACPI System Locality Information Table // (System Locality == Proximity Domain) struct SLIT { AcpiTable header; u64 numSystemLocalities; u8 entries[1]; // numSystemLocalities*numSystemLocalities entries }; #pragma pack(pop) static double ReadRelativeDistanceFromSLIT(const SLIT* slit) { const size_t n = slit->numSystemLocalities; ENSURE(slit->header.size == sizeof(SLIT)-sizeof(slit->entries)+n*n); // diagonals are specified to be 10 for(size_t i = 0; i < n; i++) ENSURE(slit->entries[i*n+i] == 10); // entries = relativeDistance * 10 return *std::max_element(slit->entries, slit->entries+n*n) / 10.0; } // @return ratio between max/min time required to access one node's // memory from each processor. static double MeasureRelativeDistance() { - const size_t size = 16*MiB; + const size_t size = 32*MiB; void* mem = page_aligned_alloc(size); + ASSUME_ALIGNED(mem, pageSize); const uintptr_t previousProcessorMask = os_cpu_SetThreadAffinityMask(os_cpu_ProcessorMask()); double minTime = 1e10, maxTime = 0.0; for(size_t node = 0; node < numa_NumNodes(); node++) { const uintptr_t processorMask = numa_ProcessorMaskFromNode(node); os_cpu_SetThreadAffinityMask(processorMask); const double startTime = timer_Time(); memset(mem, 0, size); const double elapsedTime = timer_Time() - startTime; minTime = std::min(minTime, elapsedTime); maxTime = std::max(maxTime, elapsedTime); } (void)os_cpu_SetThreadAffinityMask(previousProcessorMask); page_aligned_free(mem, size); return maxTime / minTime; } static double relativeDistance; static Status InitRelativeDistance() { // early-out for non-NUMA systems (saves some time) if(numa_NumNodes() == 1) { relativeDistance = 1.0; return INFO::OK; } // trust values reported by the BIOS, if available const SLIT* slit = (const SLIT*)acpi_GetTable("SLIT"); if(slit) relativeDistance = ReadRelativeDistanceFromSLIT(slit); else relativeDistance = MeasureRelativeDistance(); ENSURE(relativeDistance >= 1.0); - ENSURE(relativeDistance <= 3.0); // (Microsoft guideline for NUMA systems) + ENSURE(relativeDistance <= 4.0); return INFO::OK; } double numa_Factor() { static ModuleInitState initState; (void)ModuleInit(&initState, InitRelativeDistance); return relativeDistance; } static bool IsMemoryInterleaved() { if(numa_NumNodes() == 1) return false; if(!acpi_GetTable("FACP")) // no ACPI tables available return false; // indeterminate, assume not interleaved if(acpi_GetTable("SRAT")) // present iff not interleaved return false; return true; } static bool isMemoryInterleaved; static Status InitMemoryInterleaved() { isMemoryInterleaved = IsMemoryInterleaved(); return INFO::OK; } bool numa_IsMemoryInterleaved() { static ModuleInitState initState; (void)ModuleInit(&initState, InitMemoryInterleaved); return isMemoryInterleaved; } //----------------------------------------------------------------------------- // allocator // //static bool VerifyPages(void* mem, size_t size, size_t pageSize, size_t node) //{ // WUTIL_FUNC(pQueryWorkingSetEx, BOOL, (HANDLE, PVOID, DWORD)); // WUTIL_IMPORT_KERNEL32(QueryWorkingSetEx, pQueryWorkingSetEx); // if(!pQueryWorkingSetEx) // return true; // can't do anything // //#if WINVER >= 0x600 // size_t largePageSize = os_cpu_LargePageSize(); // ENSURE(largePageSize != 0); // this value is needed for later // // // retrieve attributes of all pages constituting mem // const size_t numPages = (size + pageSize-1) / pageSize; // PSAPI_WORKING_SET_EX_INFORMATION* wsi = new PSAPI_WORKING_SET_EX_INFORMATION[numPages]; // for(size_t i = 0; i < numPages; i++) // wsi[i].VirtualAddress = (u8*)mem + i*pageSize; // pQueryWorkingSetEx(GetCurrentProcess(), wsi, DWORD(sizeof(PSAPI_WORKING_SET_EX_INFORMATION)*numPages)); // // // ensure each is valid and allocated on the correct node // for(size_t i = 0; i < numPages; i++) // { // const PSAPI_WORKING_SET_EX_BLOCK& attributes = wsi[i].VirtualAttributes; // if(!attributes.Valid) // return false; // if((attributes.LargePage != 0) != (pageSize == largePageSize)) // { // debug_printf(L"NUMA: is not a large page\n"); // return false; // } // if(attributes.Node != node) // { // debug_printf(L"NUMA: allocated from remote node\n"); // return false; // } // } // // delete[] wsi; //#else // UNUSED2(mem); // UNUSED2(size); // UNUSED2(pageSize); // UNUSED2(node); //#endif // // return true; //} // // //void* numa_AllocateOnNode(size_t node, size_t size, LargePageDisposition largePageDisposition, size_t* ppageSize) //{ // ENSURE(node < numa_NumNodes()); // // // see if there will be enough memory (non-authoritative, for debug purposes only) // { // const size_t sizeMiB = size/MiB; // const size_t availableMiB = numa_AvailableMemory(node); // if(availableMiB < sizeMiB) // debug_printf(L"NUMA: warning: node reports insufficient memory (%d vs %d MB)\n", availableMiB, sizeMiB); // } // // size_t pageSize; // (used below even if ppageSize is zero) // void* const mem = numa_Allocate(size, largePageDisposition, &pageSize); // if(ppageSize) // *ppageSize = pageSize; // // // we can't use VirtualAllocExNuma - it's only available in Vista and Server 2008. // // workaround: fault in all pages now to ensure they are allocated from the // // current node, then verify page attributes. // const uintptr_t previousProcessorMask = os_cpu_SetThreadAffinityMask(numa_ProcessorMaskFromNode(node)); // memset(mem, 0, size); // (void)os_cpu_SetThreadAffinityMask(previousProcessorMask); // // VerifyPages(mem, size, pageSize, node); // // return mem; //}