Summary: ^ This avoid relying on a u8 if we're in a fbcode build
Reviewed By: edward-shen
Differential Revision: D47760411
fbshipit-source-id: 50b4dbcea51088a0a5bd8a7f0f5941c92a080938
Summary:
Original commit changeset: 9d12716a4f3d
Original Phabricator Diff: D44835399.
File thrift/annotation/cpp.thrift is not available in open source and hence OSS build fails. Will work with thrift team to fix this properly.
Reviewed By: zechengh09
Differential Revision: D45297819
fbshipit-source-id: 0bdf2f3893f9069f34707ea897c2fa8c458a704f
Summary:
Before fbcode/common/stats/ codes are open sourced , implement a basic
working version of MonotonicCounter so that all hw tests using fb303 APIs like
fb303::fbData->getStatMap() should just work
Reviewed By: jasmeetbagga
Differential Revision: D44441336
fbshipit-source-id: f3e0fe0a59523bf90dd1e99bdca7bf5eca212c58
Summary: Instead of calling out the Rust functions all the time, just use the public global variables initialised lazily.
Differential Revision: D38241744
fbshipit-source-id: dd66e8d563076fed9279358a2b22c2042d365cc9
Summary:
This diff is reverting D38078198 (8ffc0627c3)
D38078198 (8ffc0627c3) has been identified to be causing the following test or build failures:
Tests affected:
- https://www.internalfb.com/intern/test/844424997229396/
Here's the Multisect link:
https://www.internalfb.com/intern/testinfra/multisect/1093861
Here are the tasks that are relevant to this breakage:
T127584801: 5 tests started failing for oncall asic_devinfra in the last 2 weeks
We're generating a revert to back out the changes in this diff, please note the backout may land if someone accepts it.
Reviewed By: johnny90
Differential Revision: D38265484
fbshipit-source-id: e059bad0b57f59f36b5f82fc42381371321b2877
Summary: Planning on using thrift::IPPrefix as the thrift rep of folly::CIDRNetwork, similar to how we use BinaryAddress for folly::IPAddress. Adding conversions accordingly
Reviewed By: zechengh09
Differential Revision: D37901477
fbshipit-source-id: 78997131b5fb966d0b5b603202d50d61d09bd099
Summary: Last step is changing all the callsites where we make the distinction between corp and prod to take into account the new option for labs.
Reviewed By: c-ryan747
Differential Revision: D38078198
fbshipit-source-id: d71be0202b60cd26a2417c30dce1b3fdc97adf7c
Summary:
Add bundlr-sdk to rust 3p crates.
Also needed some new fixups for a couple deps, and a custom fork of bundlr to use derive_builder 0.9 which is pinned in 2 deps we use, and fixed a missing out of scope trait bug in bundlr sdk.
Reviewed By: zhuomingfb0
Differential Revision: D37962056
fbshipit-source-id: 878048840d66dffcf558351614c45f9beef2f626
Summary:
Add some stubs to replace fbcode dependencies that were breaking OSS build, plus list some necessary .cpp and libs that were missing in the CMake config
Includes small change to QsfpUtilTx.cpp to fix a -Wsequence-error that cmake flags promoted to error
Reviewed By: shri-khare
Differential Revision: D33708895
fbshipit-source-id: a4f4d557f4155d88d4c59767cd74bec95f63f312
Summary:
We are already enrolled in clang-autoformat, but not in
thrift auto format. Enable that now, so we can let linters worry about
formatting our code.
Reviewed By: phshaikh
Differential Revision: D30905313
fbshipit-source-id: bb4724cc11c455b23305e120ddcf22a3df0125b8
Summary: Adding tests to verify that NDP and ARP entry resolution happens correctly on Aggregate ports. Recently we have seen sevs where ARP resolution happens incorrectly on subports
Reviewed By: jasmeetbagga
Differential Revision: D28156706
fbshipit-source-id: 3066620e9ed0d8805cf0d2733ea6fdece47b4254
Summary: Add test cases to verify LACP behavior with local and remote link down events. For triggering remote link events, a thrift client is used.
Differential Revision: D26329897
fbshipit-source-id: fa1bdfe6df80cab77731ec6e6f21aa826aca3e7f
Summary: As titled, getting rid of Time.h and use std::chrono to compute the current time.
Differential Revision: D21005713
fbshipit-source-id: 7e9d31af4b8652199bbe36b6ef76968cadcb4f2c
Summary: As titled, fixing NowInSecFast to return current time in OSS.
Reviewed By: boryas
Differential Revision: D20996594
fbshipit-source-id: 69fb37ea3a6a01e6d0f61fb696121705da52f811
Summary:
Implementation based on what we do in bcm layer
for reporting (non histogram) port stat counters. Abstracting
this lib out so we can reuse this code in both SAI and bcm
layers. Plus pulling this out of BcmPort makes it much
easier to unit test this code.
Reviewed By: boryas
Differential Revision: D19676641
fbshipit-source-id: 88aa20f20339729f5387500bcf6084871f4e7caa
Summary:
We are now using the open source fb303 stats libraries so
these locally managed headers are no longer needed.
Reviewed By: chadaustin
Differential Revision: D16969961
fbshipit-source-id: 71df4add75b045967a0087721c0b742edcabdf4d
Summary:
All of the requisite headers are now open sourced and eden
has been converted to use them. This removes any logic that references
this directory and kills the now unused cmake file.
Reviewed By: chadaustin
Differential Revision: D16994732
fbshipit-source-id: 573b8b50540ee64590682be2a54aa94659f12368
Summary:
Most stat code in fbcode was moved in to open source under
fbcode/fb303. The only exception that we use internally is the
MonotonicCounter class. This diff moves all of our stats code to use
the open source headers and updates cmake files to pull in the open
source fb303.
Reviewed By: chadaustin, shri-khare
Differential Revision: D16969960
fbshipit-source-id: 12b7abb54c956c242c8e27eb69fd96925c7e61f7
Summary:
Drop the `2` from the `fb303_thrift_cpp2` rule name.
The old `cpp` version of fbthrift is gone, `cpp2` is the only supported C++
version, so lets drop the `2` from this build rule name.
I plan to change some of the CMake thrift rule generation code soon, to
support multiple languages (namely Python). As part of this I plan to
automatically add a language-specific suffix(`_py` or `_cpp`) to the build
rules. It seems better to use `_cpp` rather than `_cpp2` for C++ libraries.
Changing the name for this fb303 rule will make it fit this planned naming
scheme.
Reviewed By: chadaustin
Differential Revision: D16653264
fbshipit-source-id: e4da70c79ff14e9981717ab909d982770d104512
Summary: Add a stub for this so we can use it in some the Eden code.
Reviewed By: chadaustin
Differential Revision: D15831519
fbshipit-source-id: 8f16d8fcb7080d207c748281bd93b9243c104596
Summary:
D15324752, D15331361 changed includes from common/stats/* to fb303/* stats, but
this breaks the FBOSS OSS build. However, we can do without those includes.
Each of the FBOSS #include callsites either also include
- common/stats/MonotonicCounter.h (includes common/stats/ServiceData.h includes
fb303/ServiceData.h includes Exported*) or,
- common/stats/ThreadCachedServiceData.h includes
(common/stats/ThreadLocalStatsMap.h includes
common/stats/ThreadLocalStats.h includes common/stats/ServiceData.h …)
Specifically, BcmPort.h includes MonotonicCounter.h, while SwitchStats.cpp/.h,
BcmStats.cpp/.h includes ThreadCachedServiceData.h
Lastly, fix the fboss OSS stub implementation to have similar includes
Reviewed By: chadaustin
Differential Revision: D15626299
fbshipit-source-id: 540aa40150f801bb29aeadb235cb6bb1bafca6ce
Summary: Update copyright messages to match the patterns expected by our open source linters.
Reviewed By: chadaustin
Differential Revision: D15105843
fbshipit-source-id: 6c66b4e3a552f9d71e6b43f9a5e6fcda48fd00d9
Summary:
When counters are renamed (port, trunk or queues), the current FBOSS logic
creates a new monotonic counter with new name and removes (last reference to)
monotonic counter for the old name.
However, the old counter continues to unncessarily get exported. This patch
fixes it by deleting the old counters.
Differential Revision: D14942193
fbshipit-source-id: 34cf7dd5290ae87a2c8002775e9d0940615c47fb
Summary:
wedge_agent uses ThreadLocal stats update API to avoid the overhead of
acquiring lock for each stat update. Prior to this patch, wedge_agent used its
own StatsPublisher class which called publishStats every second. StatsPublisher
also asserted if startPublishThread is not running as StatsPublisher was doing
the stats publish.
However, there is no reason for wedge_agent to publish stats on its own.
Instead, the wedge_agent could use startPublishThread mechanism.
With D14255180, servicerouter/dispatcher now calls startPublishThread, which
started hitting in the aforementioned ASSERT in StatsPublisher.
This patch fixes it by having wedge_agent start the publish thread by calling
startPublishThread. Among other things, StatsPublisher also updated route stats
and port info. That functionality is moved to updateStats.
Differential Revision: D14388119
fbshipit-source-id: 503d08f12131317592be64e46243401d8a2e5bae
Summary:
This diff enables building the eden watcher by linking in thrift and
its various dependencies.
To support building in-fbsource and in the github repo, a `maybe_shipit_dir`
function is used to setup a symlink to the `eden` and `fboss/common` dirs (this
mirrors the shipit configuration for this project: we cannot simply run shipit
because we have to build on mac and windows and shipit requires Hack, and that
does not support those platforms).
I tried to persuade cmake to let me build this without the use of a symlink but
found it too difficult to teach everything about the path mapping. The
symlinks aren't terrible, but are the reason why this diff also updates some
`.gitinore` files that are seemingly unrelated to this diff.
This diff changes a couple of build/link options: without them the end product
fails to link either due to implicit/unilateral enablement of UBSAN in some of
the deps, or because warning->error promotion is turned on.
This diff includes a copy of the `ThriftCppLibrary.cmake` file from the fboss
repo. This should get centralized and shipit'ed out into the places that
consume it. That can be done when someone gets around to doing the same for
the `FindGlog.cmake` file and doesn't need to hold up this diff.
Reviewed By: simpkins
Differential Revision: D13486486
fbshipit-source-id: 3bb5b011771b2a87618147ca019b4e50a8e0aaf2
Summary:
This patch enables BST stats by default, and works as follows:
- it uses quantile stats to maintain per queue per port stats.
- these stats are exposed to ODS.
- experiments showed that a significant CPU cycles were spent in doing
bcm_cosq_bst_stat_sync which syncs the hw stat value to sw copy.
For example, calling bcm_cosq_bst_stat_sync while retrieving BST stats
for for every port/every queue/every second resulted into CPU utilization
going up from ~ 50% for Wedge 100 to ~100-120%. Thus, this patch calls
bcm_cosq_bst_stat_sync only once per minute and then retrieves all BST stats.
- The above optimization also helps when bufferstatlogging is enabled.
- when bufferstatlogging is enabled, the BST stats will be retrieved every
second for scuba logging, and quantile stats will also be exported every
second.
- The bufferstatlogging is enabled using fboss CLI (over thrift).
We no longer need enable_fine_grained_buffer_stats option. But, a new option
viz. update_bststats_interval_s is introduced that keeps the frequency of BST
stats updates configurable.
The virtual memory overhead of always maintaining BST stats is < 5%, and
physical memory overhead is < 0.1%. Thanks to the sync optimization above,
the CPU overhead is negligible. Refer Test Plan for raw data and math.
An alternative is to explore using TimeSeriesMinMax (to maintain max over a
time window) and TLCounter (to expose stats to ODS) instead of quantile stats.
One benefit of using quantile stats is that we don't need to maintain our own
TimeSeriesMinMax but can rely on common facebook wide infra. Moreoever,
quantile stats interface is (arguably) simpler than using combination of
TimeSeriesMinMax + TlCounter'.
A separate diff against configerator is sent out to expose the new bst_stats: D8902636
Differential Revision: D8764329
fbshipit-source-id: 78db3de5340497f2f5780863035dd9ff28dcd79b
Summary:
We call this function in Eden, so make it available in the fb303 stubs that
we include in the github repository.
Differential Revision: D12813738
fbshipit-source-id: 1432e751ce676410e9d2e589274c7c1dc2c45321
Summary:
Solves an issue building eden GitHub version:
eden/fs/service/EdenServer.cpp:1044:34: error: ‘class facebook::stats::ServiceData’ has no member named ‘addStatValue’
stats::ServiceData::get()->addStatValue(
^~~~~~~~~~~~
Reviewed By: bolinfest, simpkins
Differential Revision: D9029274
fbshipit-source-id: c0e5ea492c1c38f16bfae4b08d41538a2443450f
Summary:
This removes the TARGETS files from the eden github repository. The
open source buck build has been failing for several months, since buck
removed support for the thrift_library() rule.
I will potentially take a stab at adding CMake build support for Eden
at some point in the future.
Reviewed By: chadaustin
Differential Revision: D6893233
fbshipit-source-id: e6023094a807cf481ac49998c6f21b213be6c288
Summary:
aliveSince() has a misleading name and I misunderstood.
The correct implementation just returns the timestamp of when
the process started as opposed to my understanding (the number
of seconds since the process started).
Reviewed By: linhbui
Differential Revision: D6789987
fbshipit-source-id: db60bf00f6b07e31d91dfc8ed06d1c1cbe172a64
Summary:
fb303 is not open source, but fboss depends on it. So, the
open source version of fboss uses its own private, mostly
stubbed out version of fb303 (in fboss/common/...).
With the changes to QsfpCache.cpp, we now depend on aliveSince()
fb303 primitive and the open source started spewing errors
about it not being implemented.
This patch implements aliveSince() for the skeleton open source
fb303 but we'll have to figure out a better, longer term fix
(e.g., open source all of fb303).
Reviewed By: aeckert
Differential Revision: D6731321
fbshipit-source-id: 6acdcd68053e7bfcada2cfd62aaa3aecf3dec3e0
Summary:
My recent stack of diffs introduced a handful of oss bugs :(.
1. VLOG_EVERY_MS is an internal function.
fix: added an oss stub version
2. Forgot to add QsfpCache.cpp to CMakeLists.txt
fix: add it
3. vanilla gcc5.4 has a bug where it can't handle lambdas like:
[this]() {
memberFn();
}
and instead you need to write like:
[this]() {
this->memberFn();
}
fix: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67274 says it should
be fixed in gcc7, but I added 'this->' in the meantime.
4. The base fb303.thrift file we have in open source does not declare
aliveSince, which we now call on qsfp_service.
fix: add it to fb303.thrift
Reviewed By: ninas
Differential Revision: D6627705
fbshipit-source-id: 2100783df1ea0e9af0fed66e4e24ef85b71fc7e9
Summary:
This is a codemod to change from using @/ to // in basic cases.
- TARGETS files with lines starting with @/ (but excluding @/third-party:
- autodeps lines in source and TARGETS files ( (dep|manual)=@/ ), excluding @/third-party
- Targets in string macros
The only thing left of the old format should be @/third-party:foo:bar
drop-conflicts
Reviewed By: ttsugriy
Differential Revision: D6605560
fbshipit-source-id: 17d3a196b91045f0db5ee2a5afad467b6344be0b
Summary:
In this diff, the main goal is to change the old `link_state.down` to `link_state.flap`. However, the tricky part is how can we use the normalized
portName(ethX/Y/Z) instead of portX as the key name. Since, the initialization of wedge_agent will go through two stage genrally:
In such case, we need to support changing the portName for PortStats after it's created.
My last diff(D5695076) is trying to delete all the TLTimeseries of all the threads in PortUpdateHandler observer, while ThreadLocalStats will also
try to unregister(delete) the TLTimeseries before we shut down wedge_agent. Therefore, we saw this:
```
Sep 16 19:09:52 rsw1fs.17.prn2.facebook.com wedge_agent[1884398]: F0916 19:09:52.716562 1884650 ThreadLocalStats-defs.h:495] Check failed: numErased == 1 attempted to unregister a stat that was not registered
```
So I guess probably it's not a good idea to delete a TLTimeseries of some thread in a different thread, unless you can get the global lock like ThreadLocalStats
does. Thus, in this new diff, I kinda walk around it:
Differential Revision: D5914457
fbshipit-source-id: f7d47d2ececfabbab8fdc222a0f82db4184230f9
Summary:
Depend on D4492997
Currently, we assume that changes applied to HwSwitch always succeed (or the process exits). There is a case where we want to keep moving despite failures: when we get excessive number of route (probably) because of some misconfiguration. This diff is a step towards taking care of such situation.
We change the interface between SwSwitch and HwSwitch so that HwSwitch, when asked to apply delta of old and new state, returns either the new state if it succeeds, or returns a "pruned version of new state" if some elements of new state it was not able to apply.
SwSwitch now keeps track of two states, a state that has been applied in the hardware, and another one that is desired in hardware. SwSwitch strives to catch applied state to the desired state (while desired state is also changes as move state updates are arriving).
One issue with this diff: Warmboot is not preserving "unapplied" routes, i.e., unapplied routes get lost across warmboots. This will be fixed in a seperate diff.
One naming issue: I thought about keeping SwSwitch::getState() name intact, but thought that it is not a good idea to keep distinction of states opaque to callers. So, decided to change the name.
Reviewed By: ninas
Differential Revision: D4405751
fbshipit-source-id: c933b4418445ae49d5b8e21dea95e437a8e1593d
