You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-21 - 21:22

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack5slot3s.osadl.org (updated Tue Apr 21, 2026 12:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
67512246164,69sleep00-21swapper/007:09:550
65672230158,24sleep20-21swapper/207:07:332
64832201165,24sleep10-21swapper/107:06:271
64382201165,24sleep30-21swapper/307:05:533
551421430,3sleep3685699cyclictest10:45:133
6850991080,2cyclictest0-21swapper/209:45:282
6850991080,2cyclictest0-21swapper/209:45:282
68509910097,2cyclictest0-21swapper/209:32:352
94772650,2sleep00-21swapper/010:55:010
6850996157,3cyclictest0-21swapper/211:18:482
91702500,1sleep00-21swapper/007:15:010
161242480,2sleep20-21swapper/209:55:162
219792440,1sleep30-21swapper/310:10:013
6841993912,13cyclictest7790-21awk12:05:010
685699381,27cyclictest0-21swapper/312:20:313
685699381,27cyclictest0-21swapper/311:50:433
685699371,13cyclictest0-21swapper/310:35:033
6850993732,2cyclictest4614-21sed08:15:152
684699371,14cyclictest0-21swapper/110:24:071
684699371,13cyclictest0-21swapper/109:38:231
684699371,13cyclictest0-21swapper/108:57:191
684699371,11cyclictest0-21swapper/110:33:271
684699371,11cyclictest0-21swapper/109:59:351
684699361,12cyclictest0-21swapper/111:48:471
684699361,12cyclictest0-21swapper/110:18:111
684699361,11cyclictest0-21swapper/110:42:271
685699351,24cyclictest0-21swapper/311:07:473
685699351,14cyclictest0-21swapper/309:58:193
685699351,12cyclictest0-21swapper/312:09:273
685699351,12cyclictest0-21swapper/311:48:153
685699351,11cyclictest0-21swapper/312:31:593
684699351,9cyclictest0-21swapper/109:14:151
684699351,23cyclictest0-21swapper/112:33:591
684699351,14cyclictest0-21swapper/110:12:071
684699351,11cyclictest0-21swapper/111:04:031
684699351,11cyclictest0-21swapper/111:04:031
684699351,10cyclictest0-21swapper/111:27:271
684199359,13cyclictest24737-21munin-run11:30:010
685699341,14cyclictest0-21swapper/310:55:033
684699341,6cyclictest0-21swapper/112:16:271
684699341,11cyclictest0-21swapper/111:16:381
684699341,11cyclictest0-21swapper/110:48:341
684699341,10cyclictest0-21swapper/110:59:351
684699341,10cyclictest0-21swapper/110:50:141
685699331,16cyclictest0-21swapper/311:00:593
685699331,16cyclictest0-21swapper/311:00:593
685699331,14cyclictest0-21swapper/312:16:233
685699331,13cyclictest0-21swapper/310:41:233
685699331,12cyclictest0-21swapper/311:25:073
685699331,12cyclictest0-21swapper/307:56:353
685699331,11cyclictest0-21swapper/310:32:553
685699331,11cyclictest0-21swapper/310:21:593
685699331,11cyclictest0-21swapper/309:32:223
685699330,3cyclictest0-21swapper/310:27:033
685699330,13cyclictest0-21swapper/309:45:553
685699330,13cyclictest0-21swapper/309:45:543
684699334,3cyclictest0-21swapper/110:37:231
684699331,26cyclictest0-21swapper/111:40:591
684699331,15cyclictest0-21swapper/109:46:191
684699331,15cyclictest0-21swapper/109:46:191
684699331,13cyclictest0-21swapper/112:22:151
684699331,12cyclictest0-21swapper/109:21:471
684699331,10cyclictest0-21swapper/112:10:591
684699331,10cyclictest0-21swapper/112:09:391
685699321,20cyclictest0-21swapper/310:14:353
685699321,13cyclictest0-21swapper/312:36:593
685699321,13cyclictest0-21swapper/311:32:113
685699321,12cyclictest0-21swapper/309:22:273
685699320,22cyclictest0-21swapper/311:57:063
685699320,13cyclictest0-21swapper/310:00:353
684699321,9cyclictest0-21swapper/109:04:311
684699321,23cyclictest0-21swapper/111:05:111
684699321,10cyclictest0-21swapper/108:40:111
685699311,14cyclictest0-21swapper/311:35:033
685699311,13cyclictest0-21swapper/311:18:033
685699310,13cyclictest0-21swapper/309:18:463
684699315,3cyclictest0-21swapper/107:55:591
684699311,13cyclictest0-21swapper/108:07:031
684699311,12cyclictest0-21swapper/111:12:061
684699311,11cyclictest0-21swapper/111:56:231
684699311,11cyclictest0-21swapper/109:44:191
685699301,14cyclictest0-21swapper/308:04:313
685699301,12cyclictest0-21swapper/310:50:193
685699301,12cyclictest0-21swapper/309:50:423
685699300,13cyclictest0-21swapper/308:55:553
684699306,3cyclictest0-21swapper/107:43:071
684699306,3cyclictest0-21swapper/107:22:511
684699301,9cyclictest0-21swapper/111:24:071
684699301,13cyclictest0-21swapper/108:19:111
684699301,12cyclictest0-21swapper/111:53:031
684699301,11cyclictest0-21swapper/111:37:071
684699301,10cyclictest0-21swapper/112:37:311
684699301,10cyclictest0-21swapper/109:51:391
684199302,12cyclictest21939-21awk10:10:010
684199300,14cyclictest23495-21munin-run09:00:010
685699291,15cyclictest0-21swapper/309:27:273
685699291,13cyclictest27901-21diskmemload09:37:503
685699291,12cyclictest0-21swapper/312:12:193
685699291,10cyclictest24031-21memory07:45:193
685699290,13cyclictest0-21swapper/310:17:033
684699296,3cyclictest0-21swapper/108:45:111
684699296,2cyclictest0-21swapper/107:30:051
684699294,6cyclictest0-21swapper/112:29:511
6846992913,5cyclictest10894-21diskstats08:30:151
684699291,13cyclictest0-21swapper/108:22:111
684699291,12cyclictest0-21swapper/111:30:111
684699291,12cyclictest0-21swapper/107:17:391
684699291,10cyclictest0-21swapper/109:17:111
6846992910,5cyclictest7388-21users12:00:251
685699288,3cyclictest0-21swapper/309:14:183
685699281,12cyclictest0-21swapper/308:51:433
685699281,11cyclictest0-21swapper/308:39:073
685699281,11cyclictest0-21swapper/308:39:073
685699281,11cyclictest0-21swapper/307:43:353
685099280,13cyclictest0-21swapper/207:51:292
684699288,4cyclictest0-21swapper/107:51:311
684699281,11cyclictest0-21swapper/108:02:431
684699281,10cyclictest0-21swapper/109:06:311
684199281,8cyclictest9046-21kworker/0:011:04:470
684199281,8cyclictest9046-21kworker/0:011:04:470
685699278,3cyclictest0-21swapper/307:35:593
685699271,16cyclictest0-21swapper/308:46:393
684699276,2cyclictest0-21swapper/107:29:151
684699276,2cyclictest0-21swapper/107:29:151
684699273,3cyclictest0-21swapper/109:27:111
684699271,11cyclictest0-21swapper/107:36:071
684699271,10cyclictest0-21swapper/110:08:591
685699267,3cyclictest0-21swapper/312:29:233
685699267,3cyclictest0-21swapper/311:44:193
685699266,3cyclictest0-21swapper/309:03:033
6856992614,3cyclictest0-21swapper/308:40:433
685699261,12cyclictest0-21swapper/307:31:433
685699261,11cyclictest0-21swapper/309:44:513
685699261,10cyclictest0-21swapper/307:17:153
684699267,10cyclictest23463-21diskstats07:45:141
684699261,17cyclictest20592-21smart_sda08:50:231
684699261,15cyclictest0-21swapper/107:12:101
685699257,3cyclictest0-21swapper/311:23:473
685699256,3cyclictest0-21swapper/311:10:433
685699251,22cyclictest0-21swapper/307:52:473
685099255,4cyclictest7309-21grep12:00:242
684699251,10cyclictest0-21swapper/108:37:111
684699251,10cyclictest0-21swapper/108:37:111
684199252,11cyclictest16921-21idleruntime-cro08:45:010
684199251,13cyclictest10569-21strings10:55:210
685699249,3cyclictest0-21swapper/309:06:593
685699249,3cyclictest0-21swapper/308:17:153
685699245,3cyclictest0-21swapper/307:29:033
685699245,3cyclictest0-21swapper/307:29:033
6856992418,3cyclictest7047-21/usr/sbin/munin07:10:263
6856992411,3cyclictest0-21swapper/308:31:193
685699241,13cyclictest0-21swapper/308:05:313
685699241,13cyclictest0-21swapper/307:21:153
6856992410,3cyclictest0-21swapper/308:26:273
6850992418,3cyclictest808-21snmpd12:10:432
685099241,14cyclictest22531-21crond11:25:012
684699246,6cyclictest0-21swapper/110:01:311
684699241,12cyclictest0-21swapper/108:13:551
6846992410,3cyclictest9474-21pmu-power08:25:231
684199247,9cyclictest0-21swapper/009:50:250
6856992311,3cyclictest0-21swapper/312:01:463
685699231,11cyclictest808-21snmpd08:22:103
6850992320,2cyclictest0-21swapper/207:16:102
685099231,9cyclictest191650kworker/u8:209:13:402
6846992317,3cyclictest29085-21df10:25:131
684199231,10cyclictest0-21swapper/009:45:520
684199231,10cyclictest0-21swapper/009:45:520
6850992218,3cyclictest0-21swapper/208:48:592
684699221,3cyclictest5478-21memory09:30:181
684199223,10cyclictest11-21rcuc/010:00:120
685699214,3cyclictest0-21swapper/308:10:473
685099214,7cyclictest0-21swapper/212:20:122
685099211,7cyclictest25122cpuspeed_turbos09:25:112
685099211,11cyclictest13360-21/usr/sbin/munin09:50:142
684199218,3cyclictest2790-21sed10:40:010
684199216,6cyclictest0-21swapper/010:29:050
684199212,9cyclictest191650kworker/u8:209:14:530
6841992117,2cyclictest17617-21uname12:25:200
684199211,6cyclictest0-21swapper/010:32:110
6841992114,4cyclictest20955-21latency_hist12:35:000
684199211,11cyclictest21717-21head11:20:220
297082211,2sleep029941-21cstates08:00:130
685099205,4cyclictest14713-21perf08:40:012
685099205,4cyclictest14713-21perf08:40:012
685099202,9cyclictest22769-21/usr/sbin/munin11:25:232
6850992014,3cyclictest10196-21find09:40:222
685099201,10cyclictest830-21taskset09:20:162
684199209,3cyclictest21176-21sshd12:35:060
684199206,3cyclictest87350irq/126-enp1s0-08:36:260
684199206,3cyclictest87350irq/126-enp1s0-08:36:260
684199204,13cyclictest11-21rcuc/008:10:140
684199201,3cyclictest0-21swapper/011:05:230
684199201,13cyclictest0-21swapper/009:55:240
684199201,11cyclictest25603-21sendmail_mailtr10:15:230
684199201,11cyclictest0-21swapper/012:11:130
684199201,10cyclictest3014-21gltestperf09:25:140
684199201,10cyclictest20322-21grep08:50:190
149372200,2sleep20-21swapper/208:40:122
685099195,3cyclictest87650irq/129-enp1s0-12:25:202
685099195,3cyclictest87650irq/129-enp1s0-10:40:272
685099194,8cyclictest87650irq/129-enp1s0-07:40:172
685099193,4cyclictest25342-21grep10:15:192
685099191,6cyclictest12767-21grep11:00:192
685099191,6cyclictest12767-21grep11:00:192
685099191,5cyclictest0-21swapper/210:35:212
685099191,12cyclictest0-21swapper/210:45:242
684199195,3cyclictest0-21swapper/011:34:230
684199194,2cyclictest87350irq/126-enp1s0-09:24:030
684199191,7cyclictest0-21swapper/012:20:260
684199191,6cyclictest0-21swapper/010:41:160
684199191,11cyclictest0-21swapper/010:49:520
684199191,10cyclictest7270-21fschecks_time09:35:150
684199191,10cyclictest27901-21diskmemload11:17:470
684199191,10cyclictest0-21swapper/011:53:580
684199191,10cyclictest0-21swapper/009:41:130
684199190,9cyclictest0-21swapper/011:45:510
42442190,3sleep011-21rcuc/011:55:210
305742190,3sleep2685099cyclictest11:40:212
685099187,4cyclictest23160-21awk12:40:012
685099185,8cyclictest28974-40dnf11:38:082
685099185,4cyclictest18142-21sh11:15:012
685099183,4cyclictest7919-1kworker/u9:011:47:162
685099182,4cyclictest22521-21needreboot08:55:192
685099181,7cyclictest19189-21fschecks_time07:35:152
685099181,7cyclictest0-21swapper/211:05:382
685099181,7cyclictest0-21swapper/210:58:122
685099181,7cyclictest0-21swapper/210:28:272
685099181,7cyclictest0-21swapper/209:16:222
685099181,5cyclictest4220-21tail11:55:212
685099181,12cyclictest0-21swapper/211:51:512
685099181,11cyclictest0-21swapper/212:05:252
684199189,3cyclictest0-21swapper/007:36:050
684199189,2cyclictest87550irq/128-enp1s0-09:30:190
684199184,12cyclictest11-21rcuc/007:40:270
684199183,8cyclictest0-21swapper/012:15:260
684199183,3cyclictest87350irq/126-enp1s0-11:11:090
684199183,14cyclictest0-21swapper/007:35:000
684199181,3cyclictest8472-21fschecks_count12:05:150
684199181,3cyclictest11536-21strings08:30:210
6841991813,4cyclictest0-21swapper/011:38:560
6841991812,4cyclictest9053-21idleruntime08:25:180
684199181,10cyclictest7881-21sshd08:22:060
684199180,8cyclictest442-21needreboot08:05:190
3322181,2sleep20-21swapper/210:33:452
685099176,3cyclictest87350irq/126-enp1s0-10:50:222
685099175,3cyclictest9560-21processes08:25:242
685099172,2cyclictest87550irq/128-enp1s0-07:10:182
685099171,9cyclictest19129-21timerandwakeup10:00:242
685099171,9cyclictest0-21swapper/210:20:512
685099171,9cyclictest0-21swapper/210:06:442
685099171,6cyclictest25250-21cat07:50:022
685099171,6cyclictest0-21swapper/209:35:012
685099171,5cyclictest0-21swapper/211:32:502
685099171,4cyclictest1749-21/usr/sbin/munin08:10:122
6850991712,3cyclictest17291-21if_err_enp1s007:30:172
6850991712,3cyclictest10547-21/usr/sbin/munin08:30:162
685099171,15cyclictest0-21swapper/212:19:162
684199173,2cyclictest87650irq/129-enp1s0-10:20:130
684199172,3cyclictest0-21swapper/010:13:420
684199171,6cyclictest0-21swapper/009:05:190
684199171,6cyclictest0-21swapper/008:45:160
684199171,4cyclictest12134-21/usr/sbin/munin07:20:140
6841991711,3cyclictest31145-21kernelversion09:15:180
6841991710,4cyclictest26379-21ntp_states07:50:220
206762170,2sleep20-21swapper/212:31:322
685099164,3cyclictest87650irq/129-enp1s0-08:20:102
685099161,9cyclictest0-21swapper/207:20:132
685099161,6cyclictest23069-21/usr/sbin/munin10:10:182
685099161,2cyclictest0-21swapper/207:25:072
685099161,2cyclictest0-21swapper/207:25:072
6850991612,3cyclictest0-21swapper/209:02:342
6850991612,3cyclictest0-21swapper/208:02:322
685099161,14cyclictest0-21swapper/209:07:092
6850991610,3cyclictest19469-21tr08:50:122
684199169,4cyclictest24775-21ntp_kernel_pll_09:00:210
684199161,9cyclictest0-21swapper/007:26:030
684199161,9cyclictest0-21swapper/007:26:030
684199161,7cyclictest0-21swapper/007:55:140
684199161,3cyclictest29437-21cpuspeed_turbos11:40:110
6841991612,3cyclictest0-21swapper/007:46:530
684199161,12cyclictest3941-21sshd08:15:060
684199160,13cyclictest10077-21tr07:15:130
685099155,3cyclictest404-21sed08:05:192
291752150,2sleep20-21swapper/207:56:182
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional