You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-14 - 15:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Sat Mar 14, 2026 12:44:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
92482237163,24sleep20-21swapper/207:06:132
92512235161,24sleep00-21swapper/007:06:150
92322212163,38sleep10-21swapper/107:06:001
92492204166,25sleep30-21swapper/307:06:143
3231521800,5sleep0962199cyclictest11:40:130
1263221330,4sleep0962199cyclictest09:40:180
9642991061,2cyclictest0-21swapper/309:41:533
9642998784,2cyclictest0-21swapper/311:02:253
231152750,4sleep1962999cyclictest08:50:181
53052610,2sleep30-21swapper/309:25:123
9642995955,3cyclictest0-21swapper/308:02:563
100382570,4sleep341-21ksoftirqd/307:10:123
258422550,2sleep10-21swapper/111:25:131
298872500,4sleep0962199cyclictest10:20:150
63072460,3sleep10-21swapper/109:25:221
9642994541,3cyclictest0-21swapper/307:32:573
286782450,2sleep30-21swapper/310:15:243
962999331,13cyclictest3039-21ntp_states11:45:191
962999331,13cyclictest3039-21ntp_states11:45:191
962999311,13cyclictest0-21swapper/109:31:191
962199316,12cyclictest27867-21munin-run07:50:010
962199316,12cyclictest20317-21idleruntime-cro10:00:010
962199311,8cyclictest0-21swapper/010:07:450
962199311,12cyclictest0-21swapper/010:26:290
962199311,12cyclictest0-21swapper/009:09:410
962199311,12cyclictest0-21swapper/007:10:130
962999300,10cyclictest0-21swapper/112:19:031
962199301,13cyclictest0-21swapper/011:12:490
962199301,11cyclictest0-21swapper/008:05:010
67172300,5sleep36716-21diskstats11:55:123
962999291,9cyclictest0-21swapper/111:21:271
962999291,20cyclictest0-21swapper/109:41:391
962999291,12cyclictest0-21swapper/111:37:271
962199291,8cyclictest0-21swapper/010:32:410
962199291,8cyclictest0-21swapper/010:32:410
962199291,7cyclictest0-21swapper/011:45:570
962199291,7cyclictest0-21swapper/011:45:570
962199291,12cyclictest17086-21sshd12:19:250
962199291,10cyclictest27521-21basename11:30:010
962199290,27cyclictest0-21swapper/011:02:290
962999280,10cyclictest0-21swapper/112:31:191
962199281,2cyclictest0-21swapper/009:36:210
962199281,14cyclictest19913-21/usr/sbin/munin08:45:170
962199281,11cyclictest0-21swapper/011:55:570
962199281,11cyclictest0-21swapper/007:17:170
9642992720,4cyclictest30395-21ntp_states10:20:213
962999271,9cyclictest0-21swapper/108:40:511
962999271,12cyclictest0-21swapper/109:52:151
962999271,10cyclictest0-21swapper/107:14:351
962999270,8cyclictest5080-21fschecks_time08:10:151
962999270,12cyclictest0-21swapper/107:17:351
962199271,18cyclictest0-21swapper/008:30:570
962199271,16cyclictest0-21swapper/012:01:450
962199271,12cyclictest0-21swapper/012:34:090
962199271,12cyclictest0-21swapper/008:26:250
962199271,10cyclictest0-21swapper/010:00:370
962999261,9cyclictest0-21swapper/108:04:351
962999261,10cyclictest0-21swapper/112:04:431
962999261,10cyclictest0-21swapper/109:58:511
962999261,10cyclictest0-21swapper/109:00:071
962999261,10cyclictest0-21swapper/107:55:111
962999260,3cyclictest0-21swapper/109:36:231
962999260,10cyclictest0-21swapper/109:45:271
962199261,12cyclictest0-21swapper/010:56:530
962199261,10cyclictest0-21swapper/011:17:290
962199261,10cyclictest0-21swapper/009:24:090
962199261,10cyclictest0-21swapper/007:20:490
962199260,13cyclictest0-21swapper/012:14:050
962199260,11cyclictest0-21swapper/009:30:010
964299255,3cyclictest988-1kworker/u9:111:16:273
963599259,4cyclictest9099-21seq09:32:272
962999251,9cyclictest0-21swapper/108:27:511
962999251,8cyclictest496-21jbd2/dm-0-807:35:071
962999251,17cyclictest0-21swapper/108:30:251
962999251,10cyclictest0-21swapper/107:27:151
962999250,9cyclictest0-21swapper/111:06:511
962999250,12cyclictest0-21swapper/111:15:151
962199258,14cyclictest109172sleep008:25:010
962199255,3cyclictest0-21swapper/011:33:050
962199251,9cyclictest0-21swapper/008:56:290
962199251,9cyclictest0-21swapper/008:13:310
962199251,8cyclictest0-21swapper/010:35:270
962199251,12cyclictest0-21swapper/009:45:530
962199251,12cyclictest0-21swapper/009:11:250
962199251,11cyclictest0-21swapper/012:05:210
962199251,10cyclictest0-21swapper/011:39:010
962199251,10cyclictest0-21swapper/007:40:190
962199250,9cyclictest0-21swapper/011:21:050
9642992419,3cyclictest15983-21ntpq07:20:213
964299241,10cyclictest0-21swapper/311:06:253
962999241,9cyclictest0-21swapper/110:16:271
962999241,8cyclictest0-21swapper/109:24:031
962999241,8cyclictest0-21swapper/107:32:151
962999241,7cyclictest0-21swapper/108:59:471
962999241,16cyclictest17570-21sed08:40:011
962999241,15cyclictest0-21swapper/111:10:311
962999241,10cyclictest0-21swapper/107:47:031
962999240,10cyclictest0-21swapper/109:15:311
962199248,3cyclictest0-21swapper/007:57:130
962199241,9cyclictest0-21swapper/009:54:010
962199241,9cyclictest0-21swapper/007:52:170
962199241,17cyclictest0-21swapper/009:18:370
962199241,11cyclictest0-21swapper/008:07:250
962199241,11cyclictest0-21swapper/008:07:250
962199240,10cyclictest0-21swapper/012:22:410
964299239,4cyclictest851-21sssd_nss08:38:253
964299231,9cyclictest0-21swapper/310:45:163
963599232,3cyclictest16723-21cat11:05:022
963599231,7cyclictest0-21swapper/211:18:232
962999239,4cyclictest12761-21munin-run12:10:011
962999234,3cyclictest0-21swapper/110:45:071
962999231,9cyclictest0-21swapper/111:58:271
962999231,9cyclictest0-21swapper/110:23:271
962999231,8cyclictest32694-21perl11:40:151
962999231,7cyclictest0-21swapper/109:09:391
962999231,7cyclictest0-21swapper/108:18:191
962999231,4cyclictest6821-21ntp_states10:40:191
962999231,19cyclictest27854-21grep11:30:111
962999231,16cyclictest0-21swapper/110:31:011
962999231,16cyclictest0-21swapper/110:31:011
962999230,8cyclictest0-21swapper/112:11:031
962999230,7cyclictest0-21swapper/108:09:511
962999230,7cyclictest0-21swapper/108:09:511
962999230,11cyclictest0-21swapper/112:25:071
962999230,10cyclictest0-21swapper/110:26:431
962999230,10cyclictest0-21swapper/110:04:351
962199231,9cyclictest0-21swapper/009:32:370
962199231,9cyclictest0-21swapper/008:35:170
962199231,7cyclictest0-21swapper/010:19:250
962199231,16cyclictest0-21swapper/007:39:210
962199231,13cyclictest11943-21munin-node09:01:490
962199231,12cyclictest0-21swapper/007:26:290
962199231,11cyclictest0-21swapper/010:41:130
962199231,11cyclictest0-21swapper/008:41:010
962199230,11cyclictest0-21swapper/011:09:170
193772230,2sleep09-21rcu_preempt07:30:110
964299228,1cyclictest0-21swapper/309:01:253
964299221,9cyclictest0-21swapper/307:25:253
9642992218,3cyclictest0-21swapper/312:05:253
964299220,9cyclictest29243-21pmu-power07:50:223
963599221,7cyclictest0-21swapper/209:20:552
963599221,6cyclictest29220-21grep11:30:242
962999225,3cyclictest0-21swapper/107:51:051
962999223,10cyclictest0-21swapper/107:40:191
962999221,8cyclictest0-21swapper/112:37:111
962999221,8cyclictest0-21swapper/107:20:271
962999221,16cyclictest0-21swapper/111:55:011
962999221,16cyclictest0-21swapper/109:11:181
962999221,15cyclictest0-21swapper/110:54:541
962199227,3cyclictest0-21swapper/008:16:570
962199222,10cyclictest11209-21ntp_kernel_pll_10:50:210
962199221,7cyclictest0-21swapper/010:10:290
962199221,14cyclictest0-21swapper/012:39:050
962199221,14cyclictest0-21swapper/012:28:570
9621992211,3cyclictest0-21swapper/011:51:150
962199220,5cyclictest0-21swapper/010:48:010
964299218,3cyclictest0-21swapper/312:39:023
964299215,2cyclictest19479-21head09:55:223
964299212,16cyclictest12282-21tail07:15:033
964299211,6cyclictest0-21swapper/308:29:253
9642992115,3cyclictest29702-21needreboot09:05:203
9642992114,4cyclictest7362-21gltestperf08:15:163
964299211,15cyclictest0-21swapper/310:54:583
964299211,15cyclictest0-21swapper/310:11:283
964299210,11cyclictest0-21swapper/308:40:123
963599219,3cyclictest53062tr09:25:122
963599215,3cyclictest86850irq/129-enp1s0-11:35:232
963599214,4cyclictest17457-21/usr/sbin/munin11:05:172
963599212,6cyclictest0-21swapper/212:18:042
963599211,7cyclictest9672-40dnf10:46:482
963599211,7cyclictest833-21runrttasks12:11:082
963599211,7cyclictest0-21swapper/210:23:072
963599211,6cyclictest894-21sshd10:27:252
963599211,6cyclictest30686-21diskmemload11:46:532
963599211,6cyclictest30686-21diskmemload11:46:532
963599211,6cyclictest0-21swapper/209:47:332
963599211,6cyclictest0-21swapper/209:07:002
963599211,10cyclictest0-21swapper/211:26:062
963599210,6cyclictest26271-21timerandwakeup10:10:232
962999216,3cyclictest0-21swapper/108:45:371
962999211,9cyclictest0-21swapper/111:04:191
962999211,9cyclictest0-21swapper/108:24:311
962999211,8cyclictest0-21swapper/110:35:311
962999211,8cyclictest0-21swapper/110:10:271
962999211,11cyclictest23649-21memory10:05:181
962999211,10cyclictest0-21swapper/112:22:391
962999210,16cyclictest0-21swapper/110:55:101
962199210,9cyclictest0-21swapper/008:52:450
95502210,2sleep20-21swapper/209:35:142
964299201,9cyclictest0-21swapper/307:56:253
9642992014,3cyclictest15399-21grep09:45:243
964299201,3cyclictest19302-21diskstats11:10:133
9642992013,4cyclictest10611-21ntp_states09:35:233
964299201,16cyclictest27410-21unixbench_multi07:45:253
964299201,10cyclictest331-21awk10:25:243
963599206,3cyclictest86850irq/129-enp1s0-10:40:112
963599205,4cyclictest32311-21/usr/sbin/munin09:10:242
963599204,3cyclictest86850irq/129-enp1s0-12:00:252
963599202,6cyclictest0-21swapper/209:51:502
963599202,4cyclictest23069-21df10:05:122
963599202,4cyclictest0-21swapper/210:40:012
963599202,3cyclictest0-21swapper/210:55:042
963599201,7cyclictest20123-21sshd07:30:172
963599201,7cyclictest19403-21ntp_states09:55:222
963599201,7cyclictest0-21swapper/212:34:422
963599201,7cyclictest0-21swapper/211:51:222
963599201,7cyclictest0-21swapper/209:43:432
963599201,7cyclictest0-21swapper/207:28:122
963599201,6cyclictest0-21swapper/211:11:502
963599201,6cyclictest0-21swapper/210:32:352
963599201,6cyclictest0-21swapper/210:32:352
963599201,5cyclictest27647-21pmu-power09:00:212
963599201,5cyclictest24325-21needreboot11:20:192
963599201,5cyclictest11374-21processes10:50:232
963599201,5cyclictest0-21swapper/209:17:112
964299192,4cyclictest0-21swapper/311:45:213
964299192,4cyclictest0-21swapper/311:45:213
964299192,13cyclictest9593-21head08:20:163
964299191,7cyclictest0-21swapper/312:33:273
964299191,5cyclictest0-21swapper/310:08:253
964299191,15cyclictest24616-21/usr/sbin/munin07:40:193
9642991911,2cyclictest30911-21/usr/sbin/munin09:10:213
964299191,11cyclictest0-21swapper/308:55:253
964299191,10cyclictest4216-21NetworkManager08:09:013
964299191,10cyclictest4216-21NetworkManager08:09:013
964299191,10cyclictest32140-21/usr/sbin/munin11:40:233
964299190,9cyclictest0-21swapper/309:15:213
963599193,4cyclictest30637-21diskstats07:55:152
963599193,4cyclictest11887-21if_enp2s008:25:172
963599193,4cyclictest10614-21sshd08:21:252
963599193,3cyclictest86850irq/129-enp1s0-07:11:252
963599192,6cyclictest0-21swapper/210:03:322
963599192,3cyclictest32486-21/usr/sbin/munin08:00:102
963599191,7cyclictest22108-21tail07:35:172
963599191,6cyclictest0-21swapper/212:37:182
963599191,6cyclictest0-21swapper/211:41:022
963599191,6cyclictest0-21swapper/210:15:392
963599191,5cyclictest0-21swapper/212:20:022
963599191,5cyclictest0-21swapper/208:45:192
963599191,4cyclictest6752-21cpu08:15:132
963599191,4cyclictest0-21swapper/212:09:252
963599191,4cyclictest0-21swapper/211:55:102
964299185,3cyclictest86850irq/129-enp1s0-11:35:153
964299183,3cyclictest86850irq/129-enp1s0-11:30:013
964299181,9cyclictest0-21swapper/312:26:243
964299181,6cyclictest0-21swapper/312:18:053
964299181,6cyclictest0-21swapper/310:57:343
964299181,6cyclictest0-21swapper/310:37:093
964299181,6cyclictest0-21swapper/310:31:523
964299181,6cyclictest0-21swapper/310:31:523
964299181,6cyclictest0-21swapper/310:00:223
964299181,6cyclictest0-21swapper/308:30:233
9642991813,3cyclictest6079-21taskset08:10:253
964299181,15cyclictest0-21swapper/309:50:183
964299181,15cyclictest0-21swapper/309:20:323
963599184,3cyclictest86850irq/129-enp1s0-08:05:152
963599184,3cyclictest86850irq/129-enp1s0-08:05:152
963599181,7cyclictest988-1kworker/u9:112:25:262
963599181,5cyclictest16878-21latency_hist07:25:022
963599181,15cyclictest5271-21if_enp1s008:10:162
127392180,3sleep230-21rcuc/207:15:122
964299174,2cyclictest86850irq/129-enp1s0-11:30:183
964299171,6cyclictest22113-21/usr/sbin/munin08:50:133
964299171,6cyclictest0-21swapper/311:22:303
964299171,4cyclictest0-21swapper/312:20:173
964299171,4cyclictest0-21swapper/310:40:123
964299171,4cyclictest0-21swapper/307:35:183
9642991713,2cyclictest29829-21kworker/3:111:50:183
964299171,15cyclictest0-21swapper/309:32:263
964299171,14cyclictest0-21swapper/312:10:113
963599175,9cyclictest27440-21grep07:45:252
963599174,2cyclictest86750irq/128-enp1s0-08:30:272
963599172,6cyclictest0-21swapper/208:39:072
963599171,5cyclictest19212-21sshd08:40:252
963599171,13cyclictest24405-21grep08:55:122
964299162,2cyclictest86550irq/126-enp1s0-08:48:393
964299161,4cyclictest0-21swapper/312:01:473
963599161,6cyclictest0-21swapper/207:52:592
9635991612,3cyclictest0-21swapper/207:43:572
963599141,12cyclictest0-21swapper/208:50:062
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional