You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-23 - 22:28

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Mon Mar 23, 2026 12:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
202572238161,24sleep20-21swapper/207:09:022
201102225194,20sleep00-21swapper/007:07:060
201532203167,24sleep10-21swapper/107:07:391
200162201164,25sleep30-21swapper/307:05:533
1991621430,2sleep20-21swapper/208:20:112
20441991100,108cyclictest0-21swapper/308:13:273
20441991070,105cyclictest0-21swapper/311:08:223
204419910299,2cyclictest0-21swapper/308:48:063
20441999491,2cyclictest0-21swapper/310:38:083
101882810,2sleep30-21swapper/310:25:133
20441997269,2cyclictest0-21swapper/307:13:133
20441996157,3cyclictest0-21swapper/309:57:583
101002610,15sleep30-21swapper/307:55:253
20441995248,3cyclictest0-21swapper/310:13:003
96542420,1sleep20-21swapper/211:35:242
20421994014,12cyclictest16358-21perf10:40:020
76522350,11sleep07-21ksoftirqd/010:20:020
2042199359,12cyclictest17931-21perf09:30:010
2043499331,24cyclictest0-21swapper/210:57:352
2042999331,14cyclictest0-21swapper/109:10:191
2043499320,17cyclictest0-21swapper/210:10:222
2042999311,11cyclictest0-21swapper/108:40:431
2043499301,14cyclictest0-21swapper/210:40:032
2043499301,11cyclictest0-21swapper/211:48:232
2043499301,11cyclictest0-21swapper/209:09:112
2043499300,17cyclictest0-21swapper/210:49:072
2043499300,12cyclictest0-21swapper/210:06:022
2042999301,19cyclictest22961-21diskstats07:15:031
2044199291,22cyclictest26396-21kworker/3:111:14:283
2044199291,17cyclictest26396-21kworker/3:107:32:173
2043499291,12cyclictest0-21swapper/208:27:102
2043499291,10cyclictest0-21swapper/211:03:152
2043499290,12cyclictest0-21swapper/211:29:352
2043499290,12cyclictest0-21swapper/211:29:352
2042999291,21cyclictest0-21swapper/107:24:591
2042999291,20cyclictest0-21swapper/112:22:321
2042999291,13cyclictest10328-21sshd07:55:261
2042199291,16cyclictest5981-21awk11:30:000
2042199291,16cyclictest5981-21awk11:30:000
2044199284,2cyclictest26396-21kworker/3:107:45:453
2044199284,2cyclictest26396-21kworker/3:107:45:453
2044199281,11cyclictest26396-21kworker/3:109:35:093
2043499281,11cyclictest0-21swapper/210:28:022
2043499281,10cyclictest0-21swapper/209:33:232
2043499281,10cyclictest0-21swapper/207:52:272
2043499280,13cyclictest0-21swapper/212:32:022
2043499280,12cyclictest0-21swapper/212:11:062
2043499280,12cyclictest0-21swapper/208:53:592
2043499280,11cyclictest0-21swapper/211:42:312
2043499280,11cyclictest0-21swapper/211:34:152
2043499280,10cyclictest0-21swapper/207:39:272
2042999280,13cyclictest0-21swapper/107:27:151
2044199272,16cyclictest26396-21kworker/3:110:35:003
2044199272,12cyclictest26396-21kworker/3:109:16:053
2044199272,10cyclictest26396-21kworker/3:112:01:373
20441992716,2cyclictest26396-21kworker/3:111:33:253
20441992714,5cyclictest26396-21kworker/3:112:35:173
20441992714,5cyclictest26396-21kworker/3:112:35:163
2044199271,10cyclictest26396-21kworker/3:112:15:373
2044199271,10cyclictest26396-21kworker/3:109:50:373
2043499271,9cyclictest0-21swapper/207:48:232
2043499271,9cyclictest0-21swapper/207:48:232
2043499271,13cyclictest0-21swapper/211:15:472
2043499271,12cyclictest0-21swapper/211:08:432
2043499271,12cyclictest0-21swapper/210:35:022
2043499271,12cyclictest0-21swapper/209:56:582
2043499271,11cyclictest0-21swapper/212:22:502
2043499271,10cyclictest0-21swapper/208:58:392
2043499271,10cyclictest0-21swapper/208:30:342
2043499270,9cyclictest0-21swapper/207:22:592
2043499270,10cyclictest0-21swapper/212:36:422
2043499270,10cyclictest0-21swapper/212:36:422
2043499270,10cyclictest0-21swapper/207:27:352
2042999271,9cyclictest0-21swapper/111:00:231
2042999271,20cyclictest0-21swapper/109:40:301
2042999271,16cyclictest0-21swapper/112:38:431
2042999271,16cyclictest0-21swapper/112:38:431
2042999271,13cyclictest0-21swapper/108:05:341
2042999271,12cyclictest4361-21runrttasks11:25:151
2042999271,12cyclictest4361-21runrttasks11:25:151
2042999271,12cyclictest0-21swapper/108:33:031
2042199273,11cyclictest31723-21idleruntime-cro11:15:000
2044199261,9cyclictest26396-21kworker/3:111:21:133
2044199261,16cyclictest26396-21kworker/3:110:22:573
2043499261,9cyclictest0-21swapper/208:18:542
2043499261,13cyclictest0-21swapper/212:03:072
2043499261,12cyclictest0-21swapper/209:50:592
2043499261,12cyclictest0-21swapper/207:43:592
2043499261,11cyclictest0-21swapper/212:07:342
2043499261,10cyclictest0-21swapper/208:35:342
2043499260,12cyclictest0-21swapper/211:59:152
2043499260,12cyclictest0-21swapper/210:21:182
2043499260,11cyclictest0-21swapper/209:16:032
2043499260,10cyclictest0-21swapper/210:32:382
2043499260,10cyclictest0-21swapper/210:03:382
2043499260,10cyclictest0-21swapper/209:37:232
2042999261,9cyclictest0-21swapper/109:02:551
2042999261,9cyclictest0-21swapper/109:02:551
2042999261,11cyclictest0-21swapper/111:59:221
2042999261,11cyclictest0-21swapper/110:27:151
2042999261,11cyclictest0-21swapper/107:46:541
2042999261,11cyclictest0-21swapper/107:46:541
2042999261,10cyclictest0-21swapper/109:33:341
2042999261,10cyclictest0-21swapper/108:01:181
2042999260,10cyclictest0-21swapper/108:57:591
2044199252,10cyclictest26396-21kworker/3:109:28:093
2044199251,10cyclictest26396-21kworker/3:108:01:453
2043499251,8cyclictest0-21swapper/208:41:462
2043499251,11cyclictest0-21swapper/207:16:552
2043499250,12cyclictest0-21swapper/208:14:342
2043499250,11cyclictest0-21swapper/209:28:232
2043499250,11cyclictest0-21swapper/209:02:392
2043499250,11cyclictest0-21swapper/209:02:392
2043499250,10cyclictest0-21swapper/211:53:232
2042999251,9cyclictest0-21swapper/111:22:311
2042999251,9cyclictest0-21swapper/111:18:151
2042999251,8cyclictest0-21swapper/108:45:271
2042999251,17cyclictest0-21swapper/107:30:511
2042999251,12cyclictest0-21swapper/111:47:211
2042999251,12cyclictest0-21swapper/108:26:171
2042999250,12cyclictest0-21swapper/108:23:221
305312240,4sleep121-21rcuc/112:25:121
237952240,4sleep339-21rcuc/309:40:243
2044199248,5cyclictest26396-21kworker/3:110:01:573
20441992417,4cyclictest19990-21fgrep10:45:213
20441992411,3cyclictest26396-21kworker/3:111:48:573
20441992410,3cyclictest26396-21kworker/3:112:10:173
2043499241,12cyclictest0-21swapper/208:02:072
2043499240,9cyclictest0-21swapper/209:20:232
2043499240,11cyclictest0-21swapper/209:49:392
2042999248,3cyclictest0-21swapper/109:59:001
2042999245,2cyclictest496-21jbd2/dm-0-809:35:261
20429992412,6cyclictest86750irq/128-enp1s0-11:50:581
2042999241,20cyclictest860-21diskstats08:50:151
2042999241,16cyclictest0-21swapper/110:57:041
2042999241,12cyclictest812-21snmpd10:17:031
2042999241,10cyclictest0-21swapper/107:12:341
2042999240,9cyclictest0-21swapper/112:14:421
2042999240,9cyclictest0-21swapper/109:49:541
2042999240,10cyclictest0-21swapper/107:40:541
2044199239,3cyclictest26396-21kworker/3:111:16:573
2044199239,2cyclictest26396-21kworker/3:110:16:413
2044199232,11cyclictest32575-21/usr/sbin/munin12:30:223
2044199231,9cyclictest26396-21kworker/3:111:52:533
2044199231,14cyclictest0-21swapper/310:05:463
20441992310,3cyclictest26396-21kworker/3:111:40:253
20441992310,3cyclictest26396-21kworker/3:107:39:213
2043499231,10cyclictest0-21swapper/211:14:332
2043499230,9cyclictest0-21swapper/212:29:222
2043499230,12cyclictest0-21swapper/209:11:312
2043499230,10cyclictest0-21swapper/212:15:582
2042999232,8cyclictest9482-40dnf10:21:491
2042999231,9cyclictest0-21swapper/110:10:151
2042999231,9cyclictest0-21swapper/109:16:571
20429992317,3cyclictest6535-21/usr/sbin/munin07:50:111
2042999231,19cyclictest1597-21fschecks_count10:05:161
2042999231,17cyclictest0-21swapper/111:40:201
2042999231,11cyclictest0-21swapper/110:31:171
2042999231,11cyclictest0-21swapper/108:14:301
2042999231,10cyclictest0-21swapper/109:07:591
2042999230,9cyclictest0-21swapper/112:32:151
2042999230,9cyclictest0-21swapper/108:18:581
2042999230,18cyclictest0-21swapper/112:07:501
2042999230,11cyclictest0-21swapper/111:12:591
87372220,5sleep00-21swapper/011:35:140
238462220,3sleep00-21swapper/007:15:130
2044199222,7cyclictest26396-21kworker/3:111:36:493
2044199222,2cyclictest26396-21kworker/3:108:37:003
2044199221,9cyclictest26396-21kworker/3:112:25:093
2044199221,15cyclictest26396-21kworker/3:111:56:053
20441992210,3cyclictest26396-21kworker/3:111:05:003
2043499221,9cyclictest0-21swapper/208:06:192
2043499221,11cyclictest0-21swapper/210:16:382
2043499220,9cyclictest0-21swapper/207:58:352
2043499220,10cyclictest0-21swapper/209:44:472
2042999228,3cyclictest0-21swapper/110:02:501
2042999227,3cyclictest0-21swapper/111:39:221
2042999226,3cyclictest0-21swapper/112:19:421
2042999225,2cyclictest0-21swapper/110:50:191
2042999224,3cyclictest0-21swapper/109:52:541
2042999221,9cyclictest0-21swapper/111:09:411
2042999221,13cyclictest0-21swapper/110:41:021
2042999221,12cyclictest851-21sssd_nss10:45:191
2042999221,10cyclictest0-21swapper/111:34:211
2042199224,10cyclictest9122-21diskmemload11:22:280
2044199219,2cyclictest26396-21kworker/3:108:40:183
2044199212,11cyclictest26396-21kworker/3:107:43:173
2044199211,10cyclictest26396-21kworker/3:108:20:253
20441992110,3cyclictest21830-21cat12:05:113
2044199210,9cyclictest26396-21kworker/3:107:20:093
2043499210,7cyclictest0-21swapper/210:53:152
2042999215,3cyclictest0-21swapper/107:38:001
2042999215,2cyclictest9122-21diskmemload09:21:261
2042999211,8cyclictest0-21swapper/112:03:381
2042999211,14cyclictest0-21swapper/110:35:261
2042999210,9cyclictest0-21swapper/108:35:471
2042199212,11cyclictest24126-21/usr/sbin/munin08:30:250
2042199211,16cyclictest7137-21grep07:50:160
2044199209,3cyclictest26396-21kworker/3:112:21:573
2044199208,3cyclictest26396-21kworker/3:107:55:013
2044199205,6cyclictest0-21swapper/309:49:153
2044199202,8cyclictest26396-21kworker/3:108:50:283
20441992012,3cyclictest26396-21kworker/3:108:29:333
2044199201,12cyclictest0-21swapper/309:22:203
2043499206,3cyclictest0-21swapper/211:20:072
2043499206,3cyclictest0-21swapper/208:45:582
2043499201,11cyclictest0-21swapper/207:10:422
2042199204,10cyclictest812-21snmpd08:20:510
2042199201,7cyclictest0-21swapper/011:43:040
2042199201,6cyclictest0-21swapper/011:53:500
2042199201,16cyclictest29893-21awk07:30:000
2042199201,11cyclictest2416-21/usr/sbin/munin12:35:200
2042199201,11cyclictest2416-21/usr/sbin/munin12:35:200
2042199201,10cyclictest4339-21/usr/sbin/munin07:45:130
2042199201,10cyclictest4339-21/usr/sbin/munin07:45:120
323372191,2sleep032376-21ls12:30:010
2044199197,5cyclictest26396-21kworker/3:107:27:093
2044199196,3cyclictest26396-21kworker/3:107:19:123
2044199194,2cyclictest26396-21kworker/3:109:08:413
2044199191,3cyclictest0-21swapper/310:43:033
2044199191,15cyclictest0-21swapper/310:55:143
2044199191,10cyclictest26396-21kworker/3:109:10:293
2043499191,15cyclictest31974-21taskset07:33:532
2042999191,12cyclictest0-21swapper/109:25:121
2042199199,6cyclictest0-21swapper/007:44:550
2042199199,3cyclictest86750irq/128-enp1s0-10:28:180
2042199198,3cyclictest26322-21/usr/sbin/munin08:35:150
2042199198,2cyclictest32549-21sshd08:48:270
2042199197,3cyclictest12785-21/usr/sbin/munin11:45:240
2042199191,7cyclictest0-21swapper/010:23:280
2042199191,4cyclictest24134-21diskstats12:10:140
2042199191,2cyclictest86750irq/128-enp1s0-12:33:180
20421991912,4cyclictest31937-21gltestperf10:00:160
2042199191,15cyclictest2254-21ntp_states10:05:240
2042199191,12cyclictest0-21swapper/010:14:060
2042199191,10cyclictest29134-21/usr/sbin/munin09:55:230
2042199191,10cyclictest24741-21/usr/sbin/munin09:45:240
2042199191,10cyclictest17177-21/usr/sbin/munin11:55:200
277942181,2sleep00-21swapper/012:18:090
2044199186,3cyclictest0-21swapper/310:50:543
2044199182,9cyclictest26396-21kworker/3:108:57:523
2044199181,9cyclictest0-21swapper/309:30:103
2044199181,9cyclictest0-21swapper/308:30:593
2044199181,8cyclictest0-21swapper/309:00:223
2044199181,8cyclictest0-21swapper/309:00:223
2044199181,3cyclictest26396-21kworker/3:108:19:133
2044199181,14cyclictest0-21swapper/311:25:153
2044199181,14cyclictest0-21swapper/311:25:153
2042199187,3cyclictest20964-21/usr/sbin/munin10:50:240
2042199185,3cyclictest9122-21diskmemload10:45:110
2042199181,9cyclictest0-21swapper/009:12:170
2042199181,7cyclictest0-21swapper/009:00:280
2042199181,7cyclictest0-21swapper/009:00:270
2042199181,5cyclictest15844-1kworker/u9:111:04:020
2042199181,5cyclictest0-21swapper/010:44:290
2042199181,3cyclictest0-21swapper/011:31:000
20421991812,3cyclictest32146-21cut07:35:000
20421991812,3cyclictest28520-21/usr/sbin/munin08:40:130
20421991812,3cyclictest11581-21gltestperf08:00:170
20421991812,3cyclictest10704-21munin-run08:00:010
20421991811,4cyclictest22481-21perl08:25:150
2042199181,10cyclictest0-21swapper/009:44:330
22802170,2sleep00-21swapper/008:52:020
20441991712,3cyclictest13981-21latency08:05:203
2042199176,2cyclictest10686-21kworker/u8:109:23:320
2042199171,7cyclictest0-21swapper/009:17:420
2042199171,6cyclictest0-21swapper/010:59:300
2042199171,5cyclictest0-21swapper/009:33:170
2042199171,4cyclictest0-21swapper/009:37:330
2042199171,3cyclictest0-21swapper/008:16:250
2042199171,3cyclictest0-21swapper/007:35:480
20421991713,3cyclictest0-21swapper/008:13:090
20421991712,3cyclictest29253-21open_files12:20:200
20421991711,3cyclictest32279-21expr11:15:120
20421991711,3cyclictest28514-21awk09:50:260
20421991711,3cyclictest19377-21/usr/sbin/munin12:00:280
2042199171,13cyclictest0-21swapper/008:55:170
138242170,2sleep00-21swapper/010:31:060
2042199169,4cyclictest13552-21grep08:05:160
2042199161,7cyclictest0-21swapper/012:05:100
20421991612,3cyclictest0-21swapper/007:10:090
20421991611,3cyclictest25708-21/usr/sbin/munin07:20:120
20421991610,3cyclictest7776-21gltestperf09:05:160
20421991610,3cyclictest27333-21latency_hist11:05:010
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional