You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-02 - 14:29

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Tue Jun 02, 2026 12:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
35912239162,24sleep20-21swapper/207:09:332
35272202164,26sleep00-21swapper/007:08:430
34232202166,24sleep10-21swapper/107:07:231
33802201164,24sleep30-21swapper/307:06:483
404821560,5sleep3372699cyclictest10:50:013
2749221270,2sleep2371999cyclictest11:40:252
3719999491,2cyclictest0-21swapper/212:27:502
3719998884,3cyclictest0-21swapper/207:51:182
275652570,5sleep1371499cyclictest10:29:491
275652570,5sleep1371499cyclictest10:29:491
69822510,2sleep00-21swapper/010:55:160
219982470,4sleep341-21ksoftirqd/310:15:173
208832440,2sleep20-21swapper/211:25:242
192242440,1sleep20-21swapper/212:35:222
371499391,11cyclictest0-21swapper/111:42:511
278822320,1sleep30-21swapper/310:30:103
371999311,10cyclictest0-21swapper/210:31:312
371499317,10cyclictest0-21swapper/111:25:241
371999307,3cyclictest0-21swapper/209:17:102
371999307,3cyclictest0-21swapper/209:17:102
371499302,22cyclictest0-21swapper/110:34:361
3714993015,3cyclictest10684-21ntp_states08:35:221
371499292,17cyclictest87650irq/129-enp1s0-11:07:291
3714992914,9cyclictest0-21swapper/107:46:571
38492280,4sleep1221ktimersoftd/112:00:221
3714992814,9cyclictest0-21swapper/109:14:321
371499280,13cyclictest23-21ksoftirqd/111:00:551
370599282,17cyclictest19184-21kworker/0:111:35:190
370599282,17cyclictest19184-21kworker/0:111:35:190
370599281,16cyclictest877-21kworker/0:109:36:440
371499271,21cyclictest0-21swapper/109:26:141
371499270,22cyclictest0-21swapper/107:42:471
370599272,10cyclictest10480-21kworker/0:211:10:230
371999266,10cyclictest26670-21/usr/sbin/munin10:25:212
371999266,10cyclictest26670-21/usr/sbin/munin10:25:212
371999264,9cyclictest0-21swapper/212:31:272
371499264,6cyclictest0-21swapper/111:38:261
371499264,6cyclictest0-21swapper/111:38:261
371499262,15cyclictest0-21swapper/112:37:221
371499261,9cyclictest0-21swapper/111:11:251
3714992613,3cyclictest0-21swapper/109:09:341
371499261,16cyclictest0-21swapper/111:52:151
371499261,16cyclictest0-21swapper/110:10:291
371499261,16cyclictest0-21swapper/110:01:421
370599262,9cyclictest25795-21kworker/0:011:45:160
370599262,14cyclictest2668-40dnf09:30:220
85482250,4sleep230-21rcuc/209:45:152
372699258,3cyclictest0-21swapper/309:06:083
371999256,9cyclictest6978-21latency_hist12:10:012
371499253,3cyclictest0-21swapper/110:20:391
371499252,11cyclictest20733-21cpuspeed_turbos09:00:131
371499251,4cyclictest14910-21kworker/u8:211:17:131
371499251,15cyclictest0-21swapper/109:32:301
371499251,15cyclictest0-21swapper/108:59:351
371499251,15cyclictest0-21swapper/108:20:471
371499251,15cyclictest0-21swapper/107:35:121
371499251,12cyclictest23208-21awk10:20:011
371499251,12cyclictest13736-21df07:30:121
371499250,9cyclictest5510-21kworker/u8:110:07:201
371499250,16cyclictest0-21swapper/108:18:561
370599252,9cyclictest11091-21kworker/0:112:30:280
370599250,12cyclictest6241-21awk10:55:020
370599250,11cyclictest0-21swapper/008:45:010
372699248,8cyclictest0-21swapper/308:20:283
371499248,3cyclictest0-21swapper/108:15:011
371499245,9cyclictest28186-21sshd08:00:281
371499241,8cyclictest0-21swapper/112:11:221
371499241,8cyclictest0-21swapper/109:40:191
3714992413,3cyclictest0-21swapper/107:11:081
3714992412,6cyclictest0-21swapper/112:23:371
371499241,15cyclictest0-21swapper/111:56:351
371499241,15cyclictest0-21swapper/111:31:441
371499241,15cyclictest0-21swapper/110:41:351
371499241,15cyclictest0-21swapper/109:51:151
371499241,15cyclictest0-21swapper/109:20:141
371499241,15cyclictest0-21swapper/108:53:001
371499241,15cyclictest0-21swapper/107:26:321
371499241,15cyclictest0-21swapper/107:23:421
371499241,11cyclictest9390-21munin-run08:35:021
371499241,11cyclictest27555-21diskstats09:15:151
371499241,11cyclictest27555-21diskstats09:15:151
371499241,11cyclictest15469-21sed08:45:261
371499241,11cyclictest14732-21ntp_states12:25:191
371499240,17cyclictest0-21swapper/111:47:231
371499240,16cyclictest0-21swapper/112:15:191
371499240,16cyclictest0-21swapper/111:20:111
371499240,16cyclictest0-21swapper/110:35:561
370599241,8cyclictest15833-21kworker/0:208:50:270
370599241,8cyclictest10480-21kworker/0:211:23:430
370599241,12cyclictest31387-21basename09:25:000
370599241,10cyclictest8178-21kworker/0:011:02:390
211292240,3sleep341-21ksoftirqd/307:45:213
372699231,11cyclictest0-21swapper/310:35:243
371999232,11cyclictest4711-21ntp_states09:35:212
371999231,11cyclictest0-21swapper/208:20:462
371499233,8cyclictest0-21swapper/109:35:151
3714992312,3cyclictest0-21swapper/110:48:441
371499231,15cyclictest0-21swapper/110:56:331
371499231,14cyclictest0-21swapper/108:44:391
371499231,14cyclictest0-21swapper/107:51:091
371499231,11cyclictest5614-21tune2fs08:25:171
371499231,11cyclictest30849-21cat08:10:021
371499231,11cyclictest30849-21cat08:10:021
371499231,10cyclictest24782-21diskmemload12:30:201
371499230,4cyclictest30367-21kworker/u8:212:05:451
370599238,4cyclictest0-21swapper/009:55:400
370599238,13cyclictest0-21swapper/008:50:010
370599234,16cyclictest23821-21munin-run11:35:000
370599232,10cyclictest11091-21kworker/0:112:20:560
370599231,9cyclictest8839-21kworker/0:012:18:400
372699223,10cyclictest755-21rngd08:45:223
372699222,2cyclictest87450irq/127-enp1s0-09:25:223
372699221,6cyclictest0-21swapper/307:40:133
3726992211,3cyclictest27569-21needreboot08:00:193
372699221,11cyclictest19375-21df_inode10:10:133
372699221,10cyclictest32571-21vmstat08:10:233
371999222,10cyclictest24824-21timerandwakeup10:20:242
371999221,9cyclictest0-21swapper/210:18:342
371999221,7cyclictest29113-21hddtemp_smartct11:45:162
371999221,14cyclictest0-21swapper/211:15:332
371999221,12cyclictest0-21swapper/211:34:082
371999221,12cyclictest0-21swapper/209:40:212
371999221,12cyclictest0-21swapper/209:26:292
371999221,11cyclictest12263-21ntp_states07:25:202
371999221,11cyclictest0-21swapper/207:35:222
371999221,10cyclictest11251-21munin-run12:20:012
371499228,3cyclictest8474-21grep09:45:141
371499227,3cyclictest4602-21df_inode10:50:151
371499221,11cyclictest0-21swapper/107:15:201
371499221,10cyclictest25485-21open_inodes07:55:231
371499220,15cyclictest0-21swapper/109:55:471
370599222,9cyclictest28085-21kworker/0:112:04:000
370599221,9cyclictest5836-21kworker/0:009:50:320
370599221,9cyclictest23510-21kworker/0:211:41:560
370599221,6cyclictest0-21swapper/012:38:120
370599221,12cyclictest0-21swapper/010:35:010
372699216,3cyclictest24655-1kworker/u9:012:29:123
3726992114,2cyclictest0-21swapper/310:50:273
3726992112,4cyclictest0-21swapper/312:32:273
372699211,11cyclictest1481-21cpuspeed_turbos09:30:123
372699211,10cyclictest25483-21timerwakeupswit11:35:263
372699211,10cyclictest25483-21timerwakeupswit11:35:263
371999211,7cyclictest30036-21/usr/sbin/munin10:35:122
371999211,4cyclictest0-21swapper/209:55:582
371999211,13cyclictest0-21swapper/212:21:112
371999211,13cyclictest0-21swapper/207:17:482
371999211,12cyclictest0-21swapper/210:54:022
371999211,12cyclictest0-21swapper/209:30:172
371999211,11cyclictest0-21swapper/211:09:052
371999211,11cyclictest0-21swapper/209:50:502
371999211,11cyclictest0-21swapper/209:23:082
370599217,3cyclictest31824-21kworker/0:210:39:120
370599216,6cyclictest13839-21ls12:25:110
370599216,6cyclictest0-21swapper/012:06:120
370599214,15cyclictest21972-21awk07:50:010
370599213,8cyclictest551050kworker/u8:110:13:510
370599212,9cyclictest17099-21kworker/0:110:06:560
370599212,11cyclictest6517-21kworker/0:212:10:480
370599212,10cyclictest31824-21kworker/0:210:42:320
370599212,10cyclictest20787-21kworker/0:010:25:280
370599212,10cyclictest20787-21kworker/0:010:25:280
370599211,8cyclictest10680-21kworker/0:207:24:130
3705992114,4cyclictest20641-21needreboot11:25:190
370599211,3cyclictest87350irq/126-enp1s0-11:55:090
370599211,11cyclictest2056-21/usr/sbin/munin10:45:220
370599211,11cyclictest15695-21ntp_states10:00:220
3705992110,3cyclictest24782-21diskmemload09:15:140
3705992110,3cyclictest24782-21diskmemload09:15:140
250512211,16sleep325190-21cstates09:10:143
372699208,3cyclictest25009-21uptime10:20:253
372699207,3cyclictest0-21swapper/309:49:263
372699205,9cyclictest0-21swapper/309:37:233
372699201,9cyclictest8292-21processes12:10:233
372699201,8cyclictest0-21swapper/312:08:463
372699201,5cyclictest0-21swapper/309:40:173
3726992014,3cyclictest28882-21fschecks_count11:45:153
3726992013,2cyclictest0-21swapper/307:15:233
3726992012,2cyclictest0-21swapper/307:30:123
372699201,12cyclictest0-21swapper/311:44:523
372699201,11cyclictest0-21swapper/311:30:163
372699201,10cyclictest7655-21processes10:55:233
371999206,3cyclictest87550irq/128-enp1s0-08:00:182
371999203,4cyclictest2736-21ls12:00:112
371999202,6cyclictest24385-21/usr/sbin/munin07:55:222
371999202,5cyclictest18517-21ntp_states11:20:222
371999201,8cyclictest11375-21sshd08:37:282
371999201,7cyclictest31854-21/usr/sbin/munin11:50:222
371999201,11cyclictest0-21swapper/212:13:112
371999201,11cyclictest0-21swapper/211:58:102
371999201,11cyclictest0-21swapper/211:01:572
371999201,11cyclictest0-21swapper/209:05:262
371999201,11cyclictest0-21swapper/207:41:402
371999201,10cyclictest20601-21/usr/sbin/munin09:00:152
371999201,10cyclictest10840-21sshd07:23:292
371999201,10cyclictest0-21swapper/209:10:292
370599209,3cyclictest31598-21/usr/sbin/munin09:25:180
370599207,2cyclictest18787-21kworker/0:107:15:530
370599204,5cyclictest0-21swapper/010:20:180
370599202,7cyclictest5836-21kworker/0:009:49:280
370599202,5cyclictest24655-1kworker/u9:009:12:470
370599201,9cyclictest26308-21kworker/0:108:27:010
3705992012,4cyclictest0-21swapper/007:30:040
370599201,11cyclictest24385-21/usr/sbin/munin07:55:240
370599201,11cyclictest22808-21/usr/sbin/munin09:05:160
370599201,11cyclictest18406-21/usr/sbin/munin08:55:170
370599201,11cyclictest17792-21/usr/sbin/munin07:40:220
370599201,11cyclictest0-21swapper/009:41:580
372699199,3cyclictest22186-21/usr/sbin/munin07:50:243
372699195,3cyclictest32463-21kworker/3:112:00:253
372699191,5cyclictest31366-21crond09:25:013
372699191,5cyclictest30750-21latency_hist08:10:013
372699191,5cyclictest30750-21latency_hist08:10:013
3726991913,3cyclictest17749-21memory10:05:193
372699191,16cyclictest0-21swapper/309:15:203
372699191,16cyclictest0-21swapper/309:15:203
372699191,15cyclictest19653-21users12:35:263
3726991911,3cyclictest0-21swapper/311:25:283
372699191,12cyclictest0-21swapper/309:50:203
372699191,11cyclictest0-21swapper/310:25:513
372699191,11cyclictest0-21swapper/310:25:513
372699191,11cyclictest0-21swapper/309:58:143
372699191,10cyclictest9906-21gltestperf12:15:153
372699191,10cyclictest0-21swapper/310:41:263
371999195,3cyclictest87350irq/126-enp1s0-08:30:162
371999191,9cyclictest0-21swapper/211:35:072
371999191,9cyclictest0-21swapper/211:35:072
371999191,9cyclictest0-21swapper/210:45:032
371999191,9cyclictest0-21swapper/208:50:182
371999191,7cyclictest0-21swapper/210:42:522
371999191,7cyclictest0-21swapper/210:06:102
371999191,6cyclictest19379-21df10:10:132
371999191,3cyclictest0-21swapper/211:14:202
3719991911,5cyclictest5598-21sshd07:11:282
371999190,5cyclictest1-21systemd10:00:202
370599198,3cyclictest28787-21/usr/sbin/munin08:05:130
370599198,3cyclictest28787-21/usr/sbin/munin08:05:130
370599192,5cyclictest24655-1kworker/u9:011:09:200
370599192,2cyclictest0-21swapper/011:54:310
3705991913,3cyclictest26677-21tr08:00:120
370599191,10cyclictest14910-21kworker/u8:211:15:240
372699189,3cyclictest0-21swapper/308:51:293
372699185,7cyclictest0-21swapper/311:09:483
372699181,5cyclictest15741-21sed10:00:233
372699181,5cyclictest0-21swapper/311:50:283
372699181,3cyclictest8677-21cpu11:00:113
372699181,15cyclictest10554-21head07:20:263
372699181,14cyclictest13969-21memory11:10:173
372699181,11cyclictest0-21swapper/312:23:503
372699181,11cyclictest0-21swapper/311:55:163
372699181,11cyclictest0-21swapper/311:16:583
371999181,6cyclictest0-21swapper/208:07:402
371999181,6cyclictest0-21swapper/208:07:402
371999181,5cyclictest18406-21/usr/sbin/munin08:55:132
371999181,4cyclictest11812-21/usr/sbin/munin08:40:172
371999181,4cyclictest0-21swapper/210:57:062
371999181,3cyclictest0-21swapper/207:33:272
370599188,3cyclictest15597-21/usr/sbin/munin07:35:150
370599186,2cyclictest7089-21kworker/0:008:31:490
370599185,2cyclictest26308-21kworker/0:108:18:340
370599184,2cyclictest26308-21kworker/0:108:25:020
370599182,6cyclictest18058-21kworker/0:109:00:170
370599181,8cyclictest13180-21ls07:30:020
370599181,2cyclictest87350irq/126-enp1s0-10:15:130
370599180,11cyclictest0-21swapper/007:10:250
96032170,3sleep00-21swapper/008:35:030
372699172,12cyclictest7417-21/usr/sbin/munin08:30:223
3726991712,3cyclictest25864-21unixbench-2d07:55:263
372699171,14cyclictest0-21swapper/308:35:183
372699171,13cyclictest0-21swapper/308:18:073
372699171,10cyclictest0-21swapper/308:25:133
372699170,10cyclictest0-21swapper/311:21:473
371999172,13cyclictest1439-21gltestperf08:15:152
371999171,6cyclictest14028-21cpu08:45:132
371999171,5cyclictest0-21swapper/207:47:332
3719991711,3cyclictest32274-21sed08:10:212
370599173,4cyclictest26308-21kworker/0:108:12:280
372699168,5cyclictest6032-21munin-node07:15:013
372699161,13cyclictest401ktimersoftd/308:55:283
3726991610,4cyclictest12273-21ntpq07:25:213
372699160,10cyclictest0-21swapper/308:40:163
3719991610,3cyclictest5818-21gltestperf08:25:172
370599161,5cyclictest17212-21kworker/0:107:50:040
372699151,6cyclictest20601-21/usr/sbin/munin09:00:263
3726991510,2cyclictest827-21runrttasks07:35:503
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional