You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-30 - 12:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackcslot7s.osadl.org (updated Sun Aug 30, 2026 00:44:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
69978221550,3sleep33056-21PK-Backend19:50:173
72410021050,3sleep33056-21PK-Backend21:00:143
139099850,73rtkit-daemon0-21swapper/219:08:532
68404427940,13sleep00-21swapper/019:07:240
68398726642,17sleep30-21swapper/319:06:423
7415172630,4sleep10-21swapper/121:50:121
68389726140,14sleep10-21swapper/119:05:311
139099370,3rtkit-daemon1389-21rtkit-daemon21:55:200
7835232250,3sleep33056-21PK-Backend23:50:003
139099250,4rtkit-daemon1389-21rtkit-daemon00:30:361
7915502240,5sleep00-21swapper/000:10:180
684386992413,8cyclictest436-20systemd-journal22:35:013
68438499241,21cyclictest1317-21systemd21:54:462
7729202230,4sleep33056-21PK-Backend23:20:003
684386992321,1cyclictest3056-21PK-Backend23:50:143
684386992314,6cyclictest728648-21cron21:14:593
684384992321,1cyclictest0-21swapper/223:36:192
684380992321,1cyclictest0-21swapper/121:00:111
68437699231,8cyclictest692002-21awk19:30:000
139099230,14rtkit-daemon0-21swapper/219:23:312
139099230,13rtkit-daemon0-21swapper/223:15:032
7871062220,7sleep1786969-40updatedb.mlocat00:00:001
68438699223,16cyclictest781028-21open_files23:40:193
68438699222,15cyclictest756963-21apt22:35:103
684386992220,1cyclictest3056-21PK-Backend20:21:073
684386992220,1cyclictest3056-21PK-Backend00:28:203
684386992220,1cyclictest0-21swapper/320:18:223
684386992219,2cyclictest3056-21PK-Backend23:07:353
684386992213,7cyclictest726922-21cron21:10:003
68438699220,21cyclictest0-21swapper/319:40:443
68438699220,20cyclictest0-21swapper/320:25:123
68438499229,11cyclictest771104-21unin-run23:15:012
684384992220,1cyclictest0-21swapper/220:36:182
684384992220,1cyclictest0-21swapper/220:31:592
684384992220,1cyclictest0-21swapper/200:15:522
68438499220,21cyclictest0-21swapper/223:46:532
684380992220,1cyclictest0-21swapper/121:14:451
684380992220,1cyclictest0-21swapper/120:19:561
684380992220,1cyclictest0-21swapper/100:35:211
684380992219,2cyclictest3056-21PK-Backend00:16:011
684380992217,3cyclictest715221-21cut20:35:111
68437699227,14cyclictest737621-21cron21:40:000
68437699222,12cyclictest713053-21dump-pmu-power20:30:000
68437699220,20cyclictest798208-21memory00:30:170
139099220,16rtkit-daemon3056-21PK-Backend22:56:143
7543752210,1sleep2754569-21users22:25:242
684386992119,1cyclictest3056-21PK-Backend23:04:243
684386992119,1cyclictest3056-21PK-Backend21:33:193
684386992119,1cyclictest3056-21PK-Backend19:39:093
684386992119,1cyclictest0-21swapper/323:30:543
684386992119,1cyclictest0-21swapper/323:10:453
684386992119,1cyclictest0-21swapper/320:54:263
684386992119,1cyclictest0-21swapper/320:01:193
684386992119,1cyclictest0-21swapper/320:01:193
684386992115,4cyclictest744567-21turbostat.cron22:00:003
684386992115,4cyclictest684586-21apt19:10:113
68438699210,20cyclictest3056-21PK-Backend23:20:203
684384992119,1cyclictest0-21swapper/222:46:572
684384992119,1cyclictest0-21swapper/222:41:372
684384992119,1cyclictest0-21swapper/222:03:252
684384992119,1cyclictest0-21swapper/221:44:102
684384992119,1cyclictest0-21swapper/221:23:012
684384992119,1cyclictest0-21swapper/221:18:092
684384992119,1cyclictest0-21swapper/220:12:392
684384992119,1cyclictest0-21swapper/219:49:202
684384992119,1cyclictest0-21swapper/200:35:412
684384992119,1cyclictest0-21swapper/200:10:532
684384992119,1cyclictest0-21swapper/200:05:532
684384992118,2cyclictest3056-21PK-Backend21:30:052
684384992113,6cyclictest720766-21cut20:50:162
68438499210,20cyclictest0-21swapper/219:28:032
68438099212,14cyclictest736175-21runrttasks21:35:121
684380992119,1cyclictest3056-21PK-Backend22:32:101
684380992119,1cyclictest0-21swapper/123:16:051
684380992119,1cyclictest0-21swapper/123:08:071
684380992119,1cyclictest0-21swapper/123:04:081
684380992119,1cyclictest0-21swapper/122:54:291
684380992119,1cyclictest0-21swapper/122:45:371
684380992119,1cyclictest0-21swapper/122:41:511
684380992119,1cyclictest0-21swapper/121:18:511
684380992119,1cyclictest0-21swapper/120:56:201
684380992119,1cyclictest0-21swapper/120:47:271
684380992119,1cyclictest0-21swapper/120:31:381
684380992119,1cyclictest0-21swapper/120:13:211
684380992119,1cyclictest0-21swapper/120:08:071
684380992119,1cyclictest0-21swapper/119:37:581
684380992119,1cyclictest0-21swapper/119:26:431
684380992113,6cyclictest711324-21sh20:25:001
684380992113,6cyclictest688392-21sed19:20:011
68438099210,20cyclictest3056-21PK-Backend22:03:121
68438099210,20cyclictest3056-21PK-Backend00:09:251
68438099210,20cyclictest0-21swapper/123:46:541
68438099210,20cyclictest0-21swapper/121:45:481
68438099210,20cyclictest0-21swapper/121:05:261
68438099210,20cyclictest0-21swapper/119:55:081
68438099210,20cyclictest0-21swapper/119:50:431
68438099210,20cyclictest0-21swapper/119:32:371
68438099210,20cyclictest0-21swapper/100:23:101
684376992118,2cyclictest0-21swapper/022:50:140
684376992114,6cyclictest733403-21ntp_kernel_err21:25:180
684376992114,5cyclictest773053-21sshd23:20:100
68437699210,20cyclictest0-21swapper/023:19:200
68437699210,20cyclictest0-21swapper/022:26:210
139099210,3rtkit-daemon1389-21rtkit-daemon00:36:320
139099210,15rtkit-daemon3056-21PK-Backend20:09:203
684386992018,1cyclictest3056-21PK-Backend22:25:103
684386992018,1cyclictest3056-21PK-Backend22:19:293
684386992018,1cyclictest3056-21PK-Backend20:31:273
68438699200,1cyclictest3056-21PK-Backend19:30:033
68438699200,1cyclictest3056-21PK-Backend19:17:523
68438699200,19cyclictest3056-21PK-Backend22:23:013
68438699200,19cyclictest3056-21PK-Backend22:11:033
68438699200,19cyclictest3056-21PK-Backend21:52:393
68438699200,19cyclictest3056-21PK-Backend20:58:183
68438699200,19cyclictest3056-21PK-Backend19:55:313
68438699200,19cyclictest3056-21PK-Backend00:34:423
68438699200,19cyclictest3056-21PK-Backend00:01:233
68438699200,19cyclictest0-21swapper/320:42:493
68438499203,15cyclictest724393-21grep21:00:182
684384992018,1cyclictest0-21swapper/223:58:152
684384992018,1cyclictest0-21swapper/223:23:062
684384992018,1cyclictest0-21swapper/220:03:202
684384992018,1cyclictest0-21swapper/220:03:202
684384992018,1cyclictest0-21swapper/219:54:472
684384992018,1cyclictest0-21swapper/200:22:282
684384992018,1cyclictest0-21swapper/200:01:402
684384992015,4cyclictest732830-21tr21:25:122
684384992012,6cyclictest776801-21awk23:30:102
684384992012,6cyclictest757015-21awk22:35:122
684384992011,7cyclictest781014-21ntpq23:40:192
68438499200,1cyclictest3056-21PK-Backend22:25:002
68438499200,19cyclictest3056-21PK-Backend19:13:452
68438499200,19cyclictest0-21swapper/223:01:242
68438499200,19cyclictest0-21swapper/222:30:562
68438499200,19cyclictest0-21swapper/222:09:062
68438499200,19cyclictest0-21swapper/221:37:082
68438499200,19cyclictest0-21swapper/219:32:372
68438099209,9cyclictest690297-21unin-run19:25:001
68438099207,11cyclictest0-21swapper/100:30:001
684380992018,1cyclictest0-21swapper/123:51:411
684380992018,1cyclictest0-21swapper/122:27:031
684380992018,1cyclictest0-21swapper/120:51:541
684380992018,1cyclictest0-21swapper/119:13:251
684380992014,4cyclictest792160-21idleruntime-cro00:15:011
68438099200,1cyclictest3056-21PK-Backend22:35:091
68438099200,19cyclictest3056-21PK-Backend23:12:061
68438099200,19cyclictest0-21swapper/122:55:161
68438099200,19cyclictest0-21swapper/121:22:561
68438099200,19cyclictest0-21swapper/120:44:571
68438099200,19cyclictest0-21swapper/120:29:281
684376992018,1cyclictest0-21swapper/022:07:550
684376992018,1cyclictest0-21swapper/020:47:050
684376992013,6cyclictest694395-21fw_packets19:35:140
684376992013,5cyclictest794353-21expr00:20:120
684376992012,6cyclictest724055-21cat21:00:130
68437699200,1cyclictest0-21swapper/023:56:590
68437699200,19cyclictest3056-21PK-Backend20:38:050
68437699200,19cyclictest0-21swapper/023:49:480
68437699200,19cyclictest0-21swapper/023:30:430
68437699200,19cyclictest0-21swapper/022:31:270
68437699200,19cyclictest0-21swapper/022:18:300
68437699200,19cyclictest0-21swapper/021:48:560
68437699200,19cyclictest0-21swapper/021:15:100
68437699200,19cyclictest0-21swapper/021:06:240
68437699200,19cyclictest0-21swapper/020:42:240
68437699200,19cyclictest0-21swapper/020:22:040
68437699200,19cyclictest0-21swapper/020:19:200
139099200,14rtkit-daemon0-21swapper/223:06:002
139099200,14rtkit-daemon0-21swapper/220:09:352
139099200,14rtkit-daemon0-21swapper/219:38:322
139099200,13rtkit-daemon0-21swapper/221:05:432
68438699194,13cyclictest746734-21grep22:05:123
684386991917,1cyclictest3056-21PK-Backend21:40:553
684386991917,1cyclictest3056-21PK-Backend21:15:343
684386991917,1cyclictest0-21swapper/322:40:443
684386991917,1cyclictest0-21swapper/322:01:513
684386991917,1cyclictest0-21swapper/300:07:103
684386991915,3cyclictest720001-21munin-run20:50:003
684386991914,3cyclictest793502-21users00:15:233
684386991912,5cyclictest794648-21cut00:20:163
684386991912,5cyclictest790958-21expr00:10:103
684386991912,5cyclictest778707-21cut23:35:133
68438699190,1cyclictest3056-21PK-Backend21:27:043
68438699190,1cyclictest3056-21PK-Backend20:36:213
68438699190,18cyclictest3056-21PK-Backend21:22:583
68438699190,18cyclictest3056-21PK-Backend19:24:553
684384991917,1cyclictest0-21swapper/221:46:202
684384991912,5cyclictest775520-21cat23:25:162
684384991912,5cyclictest436-20systemd-journal19:41:072
68438499190,1cyclictest0-21swapper/220:42:472
68438499190,1cyclictest0-21swapper/220:26:182
68438499190,1cyclictest0-21swapper/219:16:282
68438499190,18cyclictest0-21swapper/222:19:532
68438099192,15cyclictest788702-21basename00:04:591
68438099192,13cyclictest702928-21apt20:00:091
68438099192,13cyclictest702928-21apt20:00:091
684380991917,1cyclictest3056-21PK-Backend21:56:551
684380991917,1cyclictest0-21swapper/121:25:051
684380991912,4cyclictest774782-21latency_hist23:25:011
68438099190,1cyclictest3056-21PK-Backend23:43:431
68438099190,1cyclictest3056-21PK-Backend22:06:431
68438099190,18cyclictest0-21swapper/122:24:541
684376991917,1cyclictest0-21swapper/022:00:520
684376991917,1cyclictest0-21swapper/021:52:280
684376991917,1cyclictest0-21swapper/020:07:290
684376991917,1cyclictest0-21swapper/019:58:590
684376991917,1cyclictest0-21swapper/000:03:350
684376991915,3cyclictest783834-21sort23:50:110
684376991914,3cyclictest13-21ksoftirqd/022:40:120
684376991912,5cyclictest13-21ksoftirqd/020:30:190
684376991911,6cyclictest752834-21awk22:20:220
684376991911,6cyclictest703293-21df_inode20:00:120
684376991911,6cyclictest703293-21df_inode20:00:120
68437699190,1cyclictest0-21swapper/020:55:570
68437699190,18cyclictest0-21swapper/022:10:050
139099190,13rtkit-daemon3056-21PK-Backend19:26:503
139099190,13rtkit-daemon0-21swapper/220:20:442
684384991812,5cyclictest765125-21sed22:55:222
68438099183,11cyclictest749160-21chrt22:10:221
684380991816,1cyclictest0-21swapper/121:41:421
684380991816,1cyclictest0-21swapper/119:40:221
684380991811,6cyclictest1-21systemd23:35:001
684380991811,5cyclictest775797-21ntpq23:25:191
684376991816,1cyclictest0-21swapper/022:36:000
684376991816,1cyclictest0-21swapper/020:10:480
684376991816,1cyclictest0-21swapper/000:25:320
684376991812,4cyclictest776725-21sed23:30:010
68437699180,1cyclictest0-21swapper/019:14:300
684386991715,1cyclictest3056-21PK-Backend22:47:273
684386991711,5cyclictest707910-21sed20:15:003
68438499179,6cyclictest743959-21python321:55:222
68438499178,7cyclictest728647-21apt-get21:15:002
68438499173,12cyclictest702386-21cron20:00:002
684384991715,1cyclictest0-21swapper/220:15:232
684384991715,1cyclictest0-21swapper/200:29:532
68438499170,15cyclictest749746-21cron22:14:592
684380991710,6cyclictest697659-21grep19:45:131
684380991710,6cyclictest1-21systemd23:40:001
684380991710,5cyclictest751481-21cron22:20:001
68437699179,6cyclictest1-21systemd23:10:000
684376991715,1cyclictest0-21swapper/023:02:510
684376991715,1cyclictest0-21swapper/022:56:560
684376991715,1cyclictest0-21swapper/022:46:290
684376991715,1cyclictest0-21swapper/019:31:480
684376991713,3cyclictest792626-21ls00:15:140
684376991713,3cyclictest790428-21sh00:09:590
684376991712,3cyclictest770196-21cat23:10:160
684376991711,5cyclictest720241-21cat20:50:110
684376991711,5cyclictest699105-21sed19:50:000
684376991711,4cyclictest1-21systemd19:45:000
684376991710,6cyclictest735330-21sed21:30:220
684376991710,5cyclictest732244-21apt-get21:23:110
684376991710,5cyclictest687845-21wc19:15:220
684376991710,5cyclictest436-20systemd-journal21:45:010
684386991614,1cyclictest3056-21PK-Backend23:59:473
684386991614,1cyclictest3056-21PK-Backend22:53:233
684386991613,2cyclictest3056-21PK-Backend21:38:123
684386991610,4cyclictest740438-21cut21:45:213
684384991614,1cyclictest0-21swapper/220:58:282
684384991611,4cyclictest886-21NetworkManager00:35:012
68438099167,6cyclictest736124-21latency_hist21:35:001
68437699164,2cyclictest141rcu_preempt23:45:000
684376991612,3cyclictest978-21snmpd19:20:290
684376991611,3cyclictest699567-21sh19:50:130
684376991610,4cyclictest727693-21cut21:10:160
684386991510,3cyclictest776543-21turbostat23:30:013
68438499158,6cyclictest762813-21fw_packets22:50:142
684384991412,1cyclictest0-21swapper/220:49:232
684384991410,3cyclictest978-21snmpd23:51:282
68437699143,2cyclictest141rcu_preempt23:36:020
68438699139,3cyclictest3056-21PK-Backend00:39:243
68438699130,12cyclictest3056-21PK-Backend19:46:113
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional