You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-10 - 04:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackcslot7s.osadl.org (updated Fri Apr 10, 2026 00:44:46)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
12273721320,4sleep10-21swapper/122:30:111
14904321070,3sleep00-21swapper/023:45:010
725632980,3sleep00-21swapper/020:05:110
139099760,66rtkit-daemon0-21swapper/119:06:061
5002727447,17sleep30-21swapper/319:05:113
5047526944,17sleep20-21swapper/219:05:192
5132626242,14sleep00-21swapper/019:07:530
1602442470,4sleep00-21swapper/000:15:120
139099430,3rtkit-daemon0-21swapper/022:56:590
653542360,4sleep00-21swapper/019:45:140
51712992718,6cyclictest124211-21sh22:35:003
51712992716,8cyclictest159669-21cron00:15:003
1094472270,4sleep33056-21PK-Backend21:50:153
139099260,4rtkit-daemon1389-21rtkit-daemon19:15:040
5171299252,5cyclictest0-21swapper/323:55:113
51712992515,7cyclictest436-20systemd-journal21:14:593
5169899250,23cyclictest1389-21rtkit-daemon19:30:130
1444312250,3chrt0-21swapper/323:30:123
51712992415,5cyclictest161683-21turbostat00:20:003
51707992315,6cyclictest93622-21df21:05:122
51702992321,1cyclictest0-21swapper/122:35:561
51702992320,2cyclictest3056-21PK-Backend20:25:031
5170299230,21cyclictest3056-21PK-Backend20:04:011
139099230,4rtkit-daemon1389-21rtkit-daemon19:39:410
139099230,15rtkit-daemon0-21swapper/023:22:080
881152220,3sleep30-21swapper/320:50:143
5171299222,17cyclictest142052-21cron23:25:003
51712992220,1cyclictest0-21swapper/322:50:033
51712992220,1cyclictest0-21swapper/300:38:463
51712992220,1cyclictest0-21swapper/300:31:433
51712992215,5cyclictest42-21ksoftirqd/321:05:163
51712992212,7cyclictest42-21ksoftirqd/322:45:233
5171299220,20cyclictest3056-21PK-Backend21:48:063
5171299220,20cyclictest3056-21PK-Backend19:12:583
51707992220,1cyclictest0-21swapper/223:26:212
51707992220,1cyclictest0-21swapper/222:11:182
51707992220,1cyclictest0-21swapper/219:19:012
51707992218,3cyclictest74300-21diskstats20:10:122
51707992218,3cyclictest74300-21diskstats20:10:112
51707992214,6cyclictest141448-21grep23:20:192
5170799221,2cyclictest0-21swapper/220:03:562
5170799220,2cyclictest0-21swapper/200:05:282
51702992220,1cyclictest3056-21PK-Backend20:15:021
51702992220,1cyclictest3056-21PK-Backend19:28:111
51702992220,1cyclictest0-21swapper/123:25:101
51702992220,1cyclictest0-21swapper/122:09:401
51702992219,2cyclictest3056-21PK-Backend20:37:541
5170299220,21cyclictest0-21swapper/100:12:021
5170299220,20cyclictest3056-21PK-Backend20:57:341
5170299220,20cyclictest3056-21PK-Backend19:54:001
5170299220,20cyclictest0-21swapper/121:01:191
5169899220,21cyclictest0-21swapper/023:45:020
139099220,15rtkit-daemon0-21swapper/021:08:180
139099220,14rtkit-daemon0-21swapper/022:04:290
51712992119,1cyclictest3056-21PK-Backend00:08:323
51712992119,1cyclictest0-21swapper/323:52:223
51712992119,1cyclictest0-21swapper/323:06:193
51712992119,1cyclictest0-21swapper/322:42:403
51712992119,1cyclictest0-21swapper/321:00:193
51712992119,1cyclictest0-21swapper/320:55:463
51712992119,1cyclictest0-21swapper/320:25:463
51712992119,1cyclictest0-21swapper/320:12:063
51712992119,1cyclictest0-21swapper/320:12:063
51712992119,1cyclictest0-21swapper/319:55:523
51712992118,1cyclictest3056-21PK-Backend21:24:053
5171299211,7cyclictest162687-21ntp_states00:20:213
51712992114,4cyclictest77755-21df20:20:133
51712992114,4cyclictest112060-21idleruntime-cro22:00:003
51712992112,7cyclictest118991-21ls22:20:013
51712992112,7cyclictest118991-21ls22:20:003
5171299210,20cyclictest3056-21PK-Backend21:34:063
5171299210,20cyclictest0-21swapper/320:06:523
5171299210,20cyclictest0-21swapper/300:27:173
5171299210,19cyclictest3056-21PK-Backend19:16:233
51707992119,1cyclictest0-21swapper/223:54:332
51707992119,1cyclictest0-21swapper/223:32:232
51707992119,1cyclictest0-21swapper/223:13:172
51707992119,1cyclictest0-21swapper/222:38:222
51707992119,1cyclictest0-21swapper/222:03:342
51707992119,1cyclictest0-21swapper/221:57:262
51707992119,1cyclictest0-21swapper/221:23:532
51707992119,1cyclictest0-21swapper/220:56:422
51707992119,1cyclictest0-21swapper/220:52:092
51707992119,1cyclictest0-21swapper/220:44:252
51707992119,1cyclictest0-21swapper/219:39:512
51707992119,1cyclictest0-21swapper/219:27:122
51707992119,1cyclictest0-21swapper/219:12:132
5170799210,20cyclictest0-21swapper/222:55:592
5170799210,20cyclictest0-21swapper/219:54:302
5170799210,20cyclictest0-21swapper/219:45:512
5170799210,20cyclictest0-21swapper/200:30:432
51702992119,1cyclictest3056-21PK-Backend23:41:131
51702992119,1cyclictest3056-21PK-Backend23:04:421
51702992119,1cyclictest0-21swapper/121:36:281
51702992118,2cyclictest3056-21PK-Backend19:45:361
51702992113,6cyclictest157949-21turbostat.cron00:10:001
5170299210,20cyclictest3056-21PK-Backend23:37:481
5170299210,20cyclictest3056-21PK-Backend00:40:001
5170299210,19cyclictest103430-21systemd21:35:001
51698992113,6cyclictest151865-21python323:50:220
5169899210,1cyclictest0-21swapper/021:59:300
139099210,4rtkit-daemon1389-21rtkit-daemon20:06:331
139099210,15rtkit-daemon3056-21PK-Backend23:54:391
139099210,14rtkit-daemon3056-21PK-Backend20:53:571
139099210,14rtkit-daemon0-21swapper/022:07:310
139099210,14rtkit-daemon0-21swapper/020:37:120
51712992018,1cyclictest3056-21PK-Backend23:26:133
51712992018,1cyclictest0-21swapper/323:46:073
51712992018,1cyclictest0-21swapper/320:48:163
51712992013,5cyclictest59778-21/usr/sbin/munin19:30:133
51712992013,5cyclictest124832-21df22:35:143
5171299200,1cyclictest3056-21PK-Backend22:10:493
5171299200,19cyclictest3056-21PK-Backend22:29:423
5171299200,19cyclictest3056-21PK-Backend21:42:103
5171299200,19cyclictest3056-21PK-Backend21:29:463
5171299200,19cyclictest3056-21PK-Backend21:17:243
5171299200,19cyclictest3056-21PK-Backend21:17:233
5171299200,17cyclictest58916-21python319:25:203
51707992018,1cyclictest0-21swapper/223:02:012
51707992018,1cyclictest0-21swapper/221:43:472
51707992018,1cyclictest0-21swapper/221:37:042
51707992018,1cyclictest0-21swapper/221:02:512
51707992018,1cyclictest0-21swapper/220:30:542
51707992018,1cyclictest0-21swapper/219:43:142
51707992018,1cyclictest0-21swapper/200:19:002
51707992013,5cyclictest82946-21ls20:35:142
51707992013,5cyclictest107846-21awk21:45:202
51707992011,7cyclictest72935-21ntp_states20:05:172
5170799200,1cyclictest0-21swapper/220:29:042
5170799200,19cyclictest0-21swapper/223:19:182
5170799200,19cyclictest0-21swapper/222:24:362
5170799200,19cyclictest0-21swapper/220:20:252
5170799200,19cyclictest0-21swapper/200:36:582
5170799200,19cyclictest0-21swapper/200:27:042
5170799200,19cyclictest0-21swapper/200:01:502
5170299203,15cyclictest436-20systemd-journal00:17:001
51702992018,1cyclictest3056-21PK-Backend23:56:061
51702992018,1cyclictest3056-21PK-Backend20:44:581
51702992018,1cyclictest3056-21PK-Backend20:31:521
51702992018,1cyclictest3056-21PK-Backend19:59:001
51702992018,1cyclictest0-21swapper/121:53:381
51702992018,1cyclictest0-21swapper/120:20:031
51702992018,1cyclictest0-21swapper/119:16:501
51702992016,2cyclictest110602-21apt21:55:141
51702992013,6cyclictest53360-21sshd19:12:421
5170299200,19cyclictest3056-21PK-Backend19:42:121
5170299200,19cyclictest3056-21PK-Backend00:21:411
5170299200,19cyclictest0-21swapper/123:12:191
5170299200,19cyclictest0-21swapper/122:03:321
5170299200,19cyclictest0-21swapper/121:29:111
5170299200,19cyclictest0-21swapper/121:24:521
5170299200,19cyclictest0-21swapper/121:05:521
5170299200,18cyclictest3056-21PK-Backend20:46:321
51698992018,1cyclictest0-21swapper/023:08:040
51698992018,1cyclictest0-21swapper/021:04:480
51698992018,1cyclictest0-21swapper/019:41:130
51698992018,1cyclictest0-21swapper/019:21:270
5169899200,1cyclictest3056-21PK-Backend20:10:060
5169899200,1cyclictest3056-21PK-Backend20:10:060
5169899200,19cyclictest3056-21PK-Backend23:30:220
5169899200,19cyclictest0-21swapper/021:51:120
5169899200,19cyclictest0-21swapper/021:34:020
5169899200,19cyclictest0-21swapper/000:38:490
5169899200,19cyclictest0-21swapper/000:30:580
5169899200,19cyclictest0-21swapper/000:00:100
139099200,14rtkit-daemon3056-21PK-Backend22:56:301
139099200,14rtkit-daemon0-21swapper/022:29:390
5171299199,7cyclictest883-21dbus-daemon23:40:003
5171299198,8cyclictest42-21ksoftirqd/319:35:113
5171299192,11cyclictest978-21snmpd20:15:123
51712991917,1cyclictest3056-21PK-Backend21:37:033
51712991917,1cyclictest0-21swapper/320:44:173
51712991917,1cyclictest0-21swapper/320:30:593
51712991917,1cyclictest0-21swapper/320:00:043
51712991917,1cyclictest0-21swapper/319:43:003
51712991917,1cyclictest0-21swapper/319:24:343
51712991914,3cyclictest83696-21chrt20:35:253
51712991912,5cyclictest66454-21cron19:50:003
5171299190,1cyclictest3056-21PK-Backend22:05:293
5171299190,16cyclictest149002-21awk23:45:003
51707991917,1cyclictest0-21swapper/223:08:502
51707991917,1cyclictest0-21swapper/222:53:222
51707991917,1cyclictest0-21swapper/222:09:422
51707991915,3cyclictest978-21snmpd22:40:332
51707991910,7cyclictest158210-21apt00:10:112
5170799190,1cyclictest0-21swapper/221:52:052
5170799190,1cyclictest0-21swapper/221:33:192
51702991917,1cyclictest3056-21PK-Backend23:46:261
51702991917,1cyclictest3056-21PK-Backend23:32:481
51702991917,1cyclictest3056-21PK-Backend23:08:551
51702991917,1cyclictest3056-21PK-Backend21:12:501
51702991917,1cyclictest3056-21PK-Backend00:26:081
51702991917,1cyclictest0-21swapper/122:25:361
51702991917,1cyclictest0-21swapper/122:20:011
51702991917,1cyclictest0-21swapper/121:46:081
51702991917,1cyclictest0-21swapper/121:42:231
51702991911,6cyclictest57780-21cron19:25:001
5170299190,18cyclictest0-21swapper/122:11:161
51698991917,1cyclictest0-21swapper/023:55:430
51698991917,1cyclictest0-21swapper/023:26:570
51698991917,1cyclictest0-21swapper/023:17:300
51698991917,1cyclictest0-21swapper/023:14:540
51698991917,1cyclictest0-21swapper/023:04:390
51698991917,1cyclictest0-21swapper/021:36:590
51698991917,1cyclictest0-21swapper/020:54:210
51698991917,1cyclictest0-21swapper/020:48:260
51698991917,1cyclictest0-21swapper/020:26:080
51698991917,1cyclictest0-21swapper/020:23:320
51698991917,1cyclictest0-21swapper/020:04:050
51698991917,1cyclictest0-21swapper/019:50:400
51698991917,1cyclictest0-21swapper/000:21:180
51698991917,1cyclictest0-21swapper/000:11:380
51698991917,1cyclictest0-21swapper/000:09:490
51698991916,2cyclictest0-21swapper/021:42:550
51698991913,4cyclictest2156-21gnome-shell21:47:090
5169899190,1cyclictest0-21swapper/022:45:260
5169899190,1cyclictest0-21swapper/022:16:330
5169899190,1cyclictest0-21swapper/022:16:330
5169899190,1cyclictest0-21swapper/022:12:360
5169899190,1cyclictest0-21swapper/000:29:090
5169899190,18cyclictest0-21swapper/021:27:330
5169899190,18cyclictest0-21swapper/021:11:440
5169899190,18cyclictest0-21swapper/020:41:500
5169899190,18cyclictest0-21swapper/019:14:370
5171299189,7cyclictest67093-21df19:50:123
51712991816,1cyclictest0-21swapper/323:17:353
51712991816,1cyclictest0-21swapper/323:02:543
51712991816,1cyclictest0-21swapper/322:56:053
51712991816,1cyclictest0-21swapper/300:02:163
51707991816,1cyclictest0-21swapper/223:40:142
51707991816,1cyclictest0-21swapper/220:48:242
51707991813,3cyclictest162484-21latency00:20:192
51707991812,5cyclictest61304-21wc19:35:002
51707991812,5cyclictest57341-21sed19:20:212
51707991812,5cyclictest128119-21tr22:45:152
51702991816,1cyclictest3056-21PK-Backend00:30:341
51702991816,1cyclictest3056-21PK-Backend00:04:591
51702991815,2cyclictest3056-21PK-Backend23:18:211
5169899188,8cyclictest0-21swapper/022:39:090
51698991816,1cyclictest0-21swapper/022:40:400
51698991816,1cyclictest0-21swapper/020:58:050
51698991812,5cyclictest69395-21cat19:55:220
51712991715,1cyclictest3056-21PK-Backend23:11:333
5170799178,7cyclictest146264-21cat23:35:162
51707991713,3cyclictest122455-21dump-pmu-power22:30:002
51707991711,5cyclictest69270-21cut19:55:202
51702991715,1cyclictest3056-21PK-Backend22:51:181
5170299170,4cyclictest3056-21PK-Backend22:44:341
51698991711,4cyclictest58300-21cat19:25:130
5169899171,12cyclictest122734-21apt22:30:120
5170799169,6cyclictest149637-21df23:45:132
51707991614,1cyclictest0-21swapper/220:17:012
51707991612,3cyclictest34-21ksoftirqd/200:00:012
51707991611,4cyclictest96702-21/usr/sbin/munin21:15:222
51707991611,4cyclictest96702-21/usr/sbin/munin21:15:222
51707991611,4cyclictest96018-21sed21:10:232
51707991611,4cyclictest122708-21/usr/sbin/munin22:30:122
5170299169,5cyclictest117695-21grep22:15:131
5170299169,5cyclictest117695-21grep22:15:131
5169899168,6cyclictest145959-21grep23:35:120
51698991614,1cyclictest0-21swapper/022:52:020
51698991614,1cyclictest0-21swapper/021:15:440
51698991614,1cyclictest0-21swapper/021:15:430
51698991614,1cyclictest0-21swapper/020:34:470
51698991610,5cyclictest99449-21ls21:20:230
51698991610,4cyclictest119968-21ntp_kernel_err22:20:190
139099160,13rtkit-daemon1389-21rtkit-daemon20:15:380
51712991513,1cyclictest3056-21PK-Backend22:01:443
51707991513,1cyclictest0-21swapper/221:26:512
51702991513,1cyclictest3056-21PK-Backend23:20:591
51702991513,1cyclictest3056-21PK-Backend19:34:471
51702991513,1cyclictest0-21swapper/121:15:121
51702991513,1cyclictest0-21swapper/121:15:121
51712991410,2cyclictest42-21ksoftirqd/322:25:003
5170799149,4cyclictest117704-21cut22:15:142
5170799149,4cyclictest117704-21cut22:15:132
5170299142,10cyclictest128095-21tr22:45:151
51702991311,1cyclictest3056-21PK-Backend20:14:161
51702991311,1cyclictest3056-21PK-Backend20:14:151
5170299128,3cyclictest3056-21PK-Backend19:35:201
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional