You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-04 - 10:21

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackcslot7s.osadl.org (updated Fri Sep 04, 2026 00:44:46)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
360489121080,2sleep10-21swapper/123:15:191
139099880,73rtkit-daemon0-21swapper/019:05:170
35172462830,23sleep234-21ksoftirqd/219:05:252
351757726444,14sleep30-21swapper/319:06:533
351676026241,14sleep10-21swapper/119:05:181
35804712490,1chrt3579865-21/usr/sbin/munin22:05:202
35277132440,4sleep13056-21PK-Backend19:35:131
35243332360,3sleep30-21swapper/319:25:163
351797199289,5cyclictest3056-21PK-Backend22:30:143
351797199263,13cyclictest0-21swapper/300:00:403
351797199263,13cyclictest0-21swapper/300:00:403
3517971992613,11cyclictest3558576-21unin-run21:05:003
3517971992610,14cyclictest3592121-21tr22:40:133
36271022250,3sleep13056-21PK-Backend00:20:121
35957012250,4chrt0-21swapper/122:50:171
351797199258,15cyclictest3563832-21sed21:20:003
351797199252,10cyclictest0-21swapper/300:30:033
139099250,12rtkit-daemon3056-21PK-Backend21:47:431
139099240,16rtkit-daemon3056-21PK-Backend22:55:101
351797199235,16cyclictest883-21dbus-daemon00:30:013
3517971992320,2cyclictest3056-21PK-Backend20:15:193
351797099238,13cyclictest3619780-40ogrotate00:00:002
3517970992317,4cyclictest3627166-21cat00:20:122
3517970992314,7cyclictest3529429-21/usr/sbin/munin19:40:152
351796599230,22cyclictest3056-21PK-Backend20:09:381
3517963992321,1cyclictest0-21swapper/020:13:420
139099230,16rtkit-daemon3056-21PK-Backend00:06:080
139099230,15rtkit-daemon3056-21PK-Backend20:12:191
139099230,15rtkit-daemon0-21swapper/120:30:321
3517971992220,1cyclictest0-21swapper/322:17:553
3517971992220,1cyclictest0-21swapper/320:33:313
3517971992220,1cyclictest0-21swapper/320:21:133
3517971992220,1cyclictest0-21swapper/320:06:413
3517971992220,1cyclictest0-21swapper/319:49:103
3517971992219,2cyclictest0-21swapper/319:16:183
351797199220,5cyclictest3609356-21apt23:30:153
351797099224,16cyclictest3612557-21munin-run23:40:002
3517970992220,1cyclictest0-21swapper/219:35:342
351797099220,21cyclictest0-21swapper/221:18:362
3517965992220,1cyclictest3056-21PK-Backend23:24:191
3517965992220,1cyclictest3056-21PK-Backend22:18:151
3517965992220,1cyclictest3056-21PK-Backend22:04:221
3517965992219,2cyclictest3056-21PK-Backend21:05:141
3517965992219,2cyclictest3056-21PK-Backend21:03:191
351796599220,20cyclictest3056-21PK-Backend20:00:261
3517963992220,1cyclictest0-21swapper/023:53:110
3517963992220,1cyclictest0-21swapper/023:24:040
3517963992220,1cyclictest0-21swapper/022:01:370
3517963992220,1cyclictest0-21swapper/021:27:440
3517963992220,1cyclictest0-21swapper/020:28:280
3517963992220,1cyclictest0-21swapper/019:57:130
351796399220,20cyclictest3056-21PK-Backend00:26:560
139099220,16rtkit-daemon3056-21PK-Backend23:27:091
139099220,15rtkit-daemon3056-21PK-Backend23:31:041
139099220,15rtkit-daemon3056-21PK-Backend22:39:321
139099220,15rtkit-daemon3056-21PK-Backend00:16:561
139099220,14rtkit-daemon0-21swapper/119:50:181
139099220,14rtkit-daemon0-21swapper/022:22:060
35397382210,2sleep20-21swapper/220:10:122
351797199212,17cyclictest3599768-21processes23:00:203
3517971992119,1cyclictest3056-21PK-Backend20:48:453
3517971992119,1cyclictest0-21swapper/323:45:493
3517971992119,1cyclictest0-21swapper/323:28:453
3517971992119,1cyclictest0-21swapper/322:58:173
3517971992119,1cyclictest0-21swapper/322:51:143
3517971992119,1cyclictest0-21swapper/322:09:373
3517971992119,1cyclictest0-21swapper/321:20:283
3517971992119,1cyclictest0-21swapper/321:06:573
3517971992119,1cyclictest0-21swapper/320:35:543
3517971992119,1cyclictest0-21swapper/319:39:513
3517971992119,1cyclictest0-21swapper/300:14:073
3517971992119,1cyclictest0-21swapper/300:09:063
351797199211,8cyclictest3573093-21idleruntime21:45:143
3517971992114,5cyclictest3569616-21fw_packets21:35:153
351797199210,8cyclictest3618602-21cstates23:55:143
351797199210,20cyclictest0-21swapper/323:11:003
351797199210,20cyclictest0-21swapper/322:45:333
3517970992119,1cyclictest0-21swapper/220:58:142
3517970992119,1cyclictest0-21swapper/220:33:062
3517970992117,3cyclictest0-21swapper/200:15:122
3517970992116,3cyclictest3568358-21python321:30:252
3517970992112,7cyclictest3633424-21ntp_states00:35:202
351797099210,19cyclictest0-21swapper/222:27:382
3517965992119,1cyclictest3056-21PK-Backend23:53:321
3517965992119,1cyclictest3056-21PK-Backend23:01:151
3517965992119,1cyclictest3056-21PK-Backend20:27:441
3517965992119,1cyclictest3056-21PK-Backend20:23:581
3517965992119,1cyclictest3056-21PK-Backend19:20:311
351796599210,20cyclictest0-21swapper/120:40:471
351796599210,19cyclictest3056-21PK-Backend21:56:051
351796599210,19cyclictest3056-21PK-Backend21:15:431
351796599210,19cyclictest3056-21PK-Backend20:57:311
351796599210,19cyclictest3056-21PK-Backend00:30:031
3517963992119,1cyclictest3056-21PK-Backend22:49:570
3517963992119,1cyclictest3056-21PK-Backend22:35:290
3517963992119,1cyclictest0-21swapper/022:56:590
3517963992119,1cyclictest0-21swapper/021:36:090
3517963992119,1cyclictest0-21swapper/019:46:580
3517963992119,1cyclictest0-21swapper/019:39:290
3517963992113,6cyclictest3518870-21ntp_kernel_pll_19:10:180
3517963992112,7cyclictest3599811-21python323:00:210
351796399210,20cyclictest0-21swapper/022:54:500
351796399210,20cyclictest0-21swapper/021:52:310
351796399210,20cyclictest0-21swapper/020:52:410
139099210,8rtkit-daemon0-21swapper/100:03:451
139099210,8rtkit-daemon0-21swapper/100:03:441
139099210,15rtkit-daemon3056-21PK-Backend20:07:260
139099210,15rtkit-daemon3056-21PK-Backend19:33:140
139099210,14rtkit-daemon0-21swapper/000:11:500
351797199202,5cyclictest42-21ksoftirqd/321:42:383
351797199202,5cyclictest3632679-21awk00:35:123
351797199202,5cyclictest0-21swapper/320:54:333
3517971992018,1cyclictest0-21swapper/323:07:283
3517971992018,1cyclictest0-21swapper/321:33:273
3517971992018,1cyclictest0-21swapper/320:40:413
3517971992018,1cyclictest0-21swapper/320:14:323
3517971992018,1cyclictest0-21swapper/319:40:523
3517971992018,1cyclictest0-21swapper/319:11:393
3517971992016,3cyclictest1095-21runrttasks21:27:053
3517971992013,6cyclictest3628457-21http00:25:013
3517971992013,5cyclictest3585449-21df22:20:143
351797199200,1cyclictest0-21swapper/321:55:513
351797199200,19cyclictest0-21swapper/323:43:243
351797199200,19cyclictest0-21swapper/322:14:023
351797199200,19cyclictest0-21swapper/322:14:023
351797199200,19cyclictest0-21swapper/322:01:053
351797199200,19cyclictest0-21swapper/320:57:023
351797199200,19cyclictest0-21swapper/319:55:383
351797199200,19cyclictest0-21swapper/319:32:343
3517970992018,1cyclictest0-21swapper/222:16:012
3517970992018,1cyclictest0-21swapper/221:12:142
3517970992018,1cyclictest0-21swapper/221:12:142
3517970992018,1cyclictest0-21swapper/220:53:012
3517970992018,1cyclictest0-21swapper/220:45:312
3517970992014,4cyclictest978-21snmpd21:47:022
3517970992013,5cyclictest3631432-21awk00:30:202
3517970992013,5cyclictest3533885-21seq19:50:292
351797099200,1cyclictest3056-21PK-Backend00:00:122
351797099200,1cyclictest3056-21PK-Backend00:00:122
351797099200,19cyclictest0-21swapper/223:52:002
351797099200,19cyclictest0-21swapper/223:34:232
351797099200,19cyclictest0-21swapper/223:25:512
351797099200,19cyclictest0-21swapper/222:22:232
351797099200,19cyclictest0-21swapper/221:43:312
351797099200,19cyclictest0-21swapper/220:43:352
351797099200,19cyclictest0-21swapper/220:37:532
351797099200,19cyclictest0-21swapper/220:20:502
351797099200,19cyclictest0-21swapper/219:47:452
351797099200,19cyclictest0-21swapper/219:31:562
351797099200,19cyclictest0-21swapper/219:27:292
351797099200,19cyclictest0-21swapper/200:29:122
351797099200,18cyclictest0-21swapper/222:35:152
3517965992018,1cyclictest3056-21PK-Backend23:40:561
3517965992018,1cyclictest3056-21PK-Backend23:37:111
3517965992018,1cyclictest3056-21PK-Backend20:39:591
3517965992018,1cyclictest3056-21PK-Backend19:30:321
3517965992017,2cyclictest3056-21PK-Backend21:22:321
3517965992017,2cyclictest0-21swapper/120:15:201
351796599200,1cyclictest3056-21PK-Backend22:40:331
351796599200,19cyclictest3056-21PK-Backend22:33:091
351796599200,19cyclictest3056-21PK-Backend22:23:151
351796599200,19cyclictest3056-21PK-Backend22:06:181
351796599200,19cyclictest3056-21PK-Backend21:54:091
3517963992018,1cyclictest0-21swapper/023:38:430
3517963992018,1cyclictest0-21swapper/023:18:430
3517963992018,1cyclictest0-21swapper/023:13:500
3517963992018,1cyclictest0-21swapper/022:40:580
3517963992018,1cyclictest0-21swapper/022:06:500
3517963992018,1cyclictest0-21swapper/020:56:340
3517963992018,1cyclictest0-21swapper/020:04:300
3517963992018,1cyclictest0-21swapper/019:19:330
3517963992017,2cyclictest3056-21PK-Backend21:10:540
3517963992017,2cyclictest3056-21PK-Backend21:10:530
351796399200,1cyclictest3056-21PK-Backend20:15:510
351796399200,1cyclictest0-21swapper/020:35:250
351796399200,1cyclictest0-21swapper/019:28:470
351796399200,19cyclictest0-21swapper/023:25:380
351796399200,19cyclictest0-21swapper/021:46:080
351796399200,19cyclictest0-21swapper/000:35:550
139099200,14rtkit-daemon3056-21PK-Backend23:56:471
139099200,13rtkit-daemon0-21swapper/022:34:470
139099200,13rtkit-daemon0-21swapper/021:02:100
351797199194,13cyclictest3616866-21memory23:50:213
3517971991917,1cyclictest0-21swapper/323:38:043
3517971991917,1cyclictest0-21swapper/300:18:203
3517971991916,2cyclictest0-21swapper/321:50:023
351797199191,5cyclictest42-21ksoftirqd/321:10:313
351797199191,5cyclictest42-21ksoftirqd/321:10:313
3517971991913,4cyclictest3522298-21cpuspeed_turbos19:20:123
351797199190,1cyclictest0-21swapper/323:17:163
351797199190,1cyclictest0-21swapper/322:25:393
351797199190,1cyclictest0-21swapper/320:00:463
351797199190,1cyclictest0-21swapper/319:52:413
3517970991917,1cyclictest3056-21PK-Backend23:47:272
3517970991917,1cyclictest0-21swapper/223:16:452
3517970991917,1cyclictest0-21swapper/223:04:012
3517970991917,1cyclictest0-21swapper/221:52:362
3517970991917,1cyclictest0-21swapper/220:08:262
3517970991917,1cyclictest0-21swapper/219:56:372
3517970991917,1cyclictest0-21swapper/219:14:532
3517970991917,1cyclictest0-21swapper/200:10:112
3517970991913,5cyclictest3557463-21forks21:00:162
3517970991912,5cyclictest3595018-21sshd22:47:412
3517970991912,5cyclictest3581960-21expr22:10:172
3517970991912,5cyclictest3581960-21expr22:10:162
3517970991911,6cyclictest3603491-21awk23:10:252
3517970991911,6cyclictest3536993-21ntp_states20:00:182
351797099190,1cyclictest0-21swapper/223:22:402
351797099190,1cyclictest0-21swapper/220:25:232
351797099190,18cyclictest0-21swapper/222:58:202
3517965991917,1cyclictest3056-21PK-Backend21:39:421
3517965991916,2cyclictest3056-21PK-Backend23:11:351
3517965991916,2cyclictest3056-21PK-Backend19:47:271
3517965991916,2cyclictest3056-21PK-Backend19:27:071
351796599190,1cyclictest3056-21PK-Backend21:44:421
351796599190,1cyclictest3056-21PK-Backend21:34:071
351796599190,1cyclictest3056-21PK-Backend19:56:071
351796599190,18cyclictest3056-21PK-Backend21:14:201
351796599190,18cyclictest3056-21PK-Backend21:14:201
351796599190,18cyclictest3056-21PK-Backend20:53:051
3517963991917,1cyclictest3056-21PK-Backend23:48:380
3517963991917,1cyclictest0-21swapper/022:10:090
3517963991917,1cyclictest0-21swapper/022:10:080
3517963991917,1cyclictest0-21swapper/020:40:120
3517963991917,1cyclictest0-21swapper/020:21:460
3517963991917,1cyclictest0-21swapper/019:24:340
3517963991916,2cyclictest0-21swapper/022:27:180
3517963991916,2cyclictest0-21swapper/019:40:300
3517963991914,3cyclictest3568951-21dump-pmu-power21:35:010
3517963991911,6cyclictest3583740-21forks22:15:150
351796399190,1cyclictest0-21swapper/023:43:170
351796399190,1cyclictest0-21swapper/021:56:230
139099190,13rtkit-daemon0-21swapper/000:16:420
139099190,12rtkit-daemon0-21swapper/021:18:360
3517971991812,5cyclictest3605864-21basename23:20:123
351797199180,15cyclictest3590272-21/usr/sbin/munin22:35:113
3517970991816,1cyclictest0-21swapper/222:31:102
3517970991816,1cyclictest0-21swapper/219:24:542
3517970991811,5cyclictest913-21gdbus21:23:442
3517970991811,5cyclictest3595478-21find22:50:162
3517970991811,5cyclictest3570411-21sshd21:36:412
3517970991810,6cyclictest883-21dbus-daemon21:55:402
3517970991810,6cyclictest3601450-21awk23:05:182
3517970991810,5cyclictest3623384-21latency_hist00:10:002
3517965991816,1cyclictest3056-21PK-Backend20:47:301
3517965991816,1cyclictest3056-21PK-Backend19:42:001
3517965991816,1cyclictest3056-21PK-Backend19:15:451
351796599180,1cyclictest3056-21PK-Backend23:49:471
351796599180,1cyclictest3056-21PK-Backend23:08:041
351796599180,1cyclictest3056-21PK-Backend19:10:501
3517963991816,1cyclictest0-21swapper/023:59:450
3517963991816,1cyclictest0-21swapper/023:33:220
3517963991816,1cyclictest0-21swapper/023:08:550
3517963991815,2cyclictest3056-21PK-Backend20:30:050
3517963991810,3cyclictest3559719-21switchtime21:05:220
351796399180,1cyclictest0-21swapper/021:41:420
351797199179,7cyclictest3545394-21cat20:25:163
3517970991715,1cyclictest0-21swapper/222:01:072
3517970991712,3cyclictest3613696-21switchtime23:40:222
351797099171,13cyclictest3542328-21python320:15:222
3517970991710,5cyclictest3560264-21cron21:09:592
351796599170,1cyclictest3056-21PK-Backend22:13:151
351796599170,1cyclictest3056-21PK-Backend22:13:151
351796399179,7cyclictest883-21dbus-daemon21:23:330
351797099168,6cyclictest3566338-21memory21:25:192
351797099161,14cyclictest1095-21runrttasks19:19:372
3517970991611,3cyclictest3592388-21sh22:40:142
3517965991614,1cyclictest3056-21PK-Backend22:27:071
3517965991614,1cyclictest0-21swapper/100:26:511
351796399168,6cyclictest3620030-21awk00:00:110
351796399168,6cyclictest3620030-21awk00:00:110
3517963991614,1cyclictest0-21swapper/020:47:070
351796599157,6cyclictest3594202-21cat22:45:151
3517965991510,3cyclictest3634171-21munin-plugin-st00:40:011
351796599150,5cyclictest3056-21PK-Backend00:10:221
3517963991511,3cyclictest3056-21PK-Backend19:51:520
139099150,3rtkit-daemon3056-21PK-Backend00:08:381
351796599149,4cyclictest3565741-21cat21:25:131
351796399149,3cyclictest3627828-21python300:20:210
3517963991310,2cyclictest3631112-21tr00:30:150
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional