You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-21 - 20:31

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Sat Mar 21, 2026 12:44:46)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
162308722560,7sleep30-21swapper/307:05:173
162863821230,3sleep13056-21PK-Backend07:20:121
170314321100,3sleep234-21ksoftirqd/210:50:162
162407927451,17sleep00-21swapper/007:09:530
162351326646,13sleep20-21swapper/207:05:252
162396726240,14sleep10-21swapper/107:08:251
17419612510,3sleep10-21swapper/112:35:211
16516292440,4sleep13056-21PK-Backend08:25:161
16779822410,3sleep33056-21PK-Backend09:40:043
16631332320,8sleep234-21ksoftirqd/209:00:002
162423299284,22cyclictest1672283-21perf09:25:001
162423299270,25cyclictest1707840-21apt-get11:03:411
1624232992619,6cyclictest26-21ksoftirqd/112:24:411
1624232992619,4cyclictest1680611-21python309:45:211
16708272250,2sleep33056-21PK-Backend09:20:103
162424599257,17cyclictest1652692-21cron08:30:003
1624245992419,3cyclictest1701334-21df10:45:123
1624232992414,8cyclictest1735166-21apt-get12:19:591
139099240,3rtkit-daemon1389-21rtkit-daemon10:00:292
162424599239,9cyclictest883-21dbus-daemon11:46:123
1624245992320,2cyclictest3056-21PK-Backend09:50:233
1624232992314,7cyclictest2636-21xdg-desktop-por10:05:081
1624232992312,9cyclictest883-21dbus-daemon10:25:171
17301842220,3sleep23056-21PK-Backend12:05:102
17267132220,3sleep3411rcuc/311:55:133
1624245992220,1cyclictest3056-21PK-Backend09:09:483
1624245992220,1cyclictest0-21swapper/311:10:143
1624245992220,1cyclictest0-21swapper/310:40:403
1624245992219,2cyclictest3056-21PK-Backend10:12:553
1624245992219,2cyclictest3056-21PK-Backend09:55:173
1624245992219,2cyclictest3056-21PK-Backend09:02:043
1624245992214,6cyclictest1691592-21sshd10:15:533
1624245992213,7cyclictest1640312-21munin-run07:55:003
1624245992212,7cyclictest2156-21gnome-shell11:03:523
162424599220,21cyclictest3056-21PK-Backend11:44:263
1624237992220,1cyclictest0-21swapper/211:32:522
1624237992219,2cyclictest0-21swapper/208:01:272
1624237992215,5cyclictest34-21ksoftirqd/211:02:092
162423799220,21cyclictest0-21swapper/209:54:572
162423799220,20cyclictest0-21swapper/211:13:052
1624232992220,1cyclictest0-21swapper/111:12:131
1624232992220,1cyclictest0-21swapper/110:03:531
1624232992220,1cyclictest0-21swapper/108:14:091
1624232992219,2cyclictest3056-21PK-Backend08:36:391
1624232992217,3cyclictest1692623-21fw_packets10:20:161
1624232992214,6cyclictest26-21ksoftirqd/112:06:291
162423299220,20cyclictest3056-21PK-Backend08:20:521
1624229992214,6cyclictest1697240-21/usr/sbin/munin10:35:010
1624229992212,8cyclictest1-21systemd12:29:540
162422999220,3cyclictest1711147-21python311:10:190
1624245992119,1cyclictest3056-21PK-Backend11:39:063
1624245992119,1cyclictest3056-21PK-Backend10:33:503
1624245992119,1cyclictest3056-21PK-Backend09:16:093
1624245992119,1cyclictest0-21swapper/311:51:033
1624245992119,1cyclictest0-21swapper/310:20:043
1624245992119,1cyclictest0-21swapper/307:26:123
1624245992119,1cyclictest0-21swapper/307:18:353
1624245992118,2cyclictest3056-21PK-Backend12:34:223
1624245992118,2cyclictest3056-21PK-Backend10:29:163
1624245992118,2cyclictest3056-21PK-Backend09:47:463
1624245992118,2cyclictest3056-21PK-Backend09:37:193
1624245992118,1cyclictest3056-21PK-Backend11:20:483
1624245992114,5cyclictest42-21ksoftirqd/310:56:063
162424599210,20cyclictest3056-21PK-Backend12:20:303
162424599210,20cyclictest3056-21PK-Backend10:07:483
162424599210,19cyclictest3056-21PK-Backend10:39:253
162424599210,19cyclictest3056-21PK-Backend08:41:153
162424599210,19cyclictest3056-21PK-Backend08:06:413
162424599210,19cyclictest3056-21PK-Backend08:01:003
1624237992119,1cyclictest0-21swapper/212:27:412
1624237992119,1cyclictest0-21swapper/212:12:462
1624237992119,1cyclictest0-21swapper/211:20:072
1624237992119,1cyclictest0-21swapper/209:46:052
1624237992119,1cyclictest0-21swapper/209:30:362
1624237992119,1cyclictest0-21swapper/208:54:552
1624237992119,1cyclictest0-21swapper/208:27:022
1624237992119,1cyclictest0-21swapper/208:15:482
1624237992119,1cyclictest0-21swapper/208:08:312
1624237992119,1cyclictest0-21swapper/207:39:522
1624237992119,1cyclictest0-21swapper/207:15:252
1624237992114,5cyclictest1691682-21cron10:17:012
1624237992111,8cyclictest883-21dbus-daemon10:55:552
162423799210,20cyclictest978-21snmpd09:14:562
162423799210,20cyclictest0-21swapper/210:10:192
162423799210,20cyclictest0-21swapper/207:40:402
162423799210,19cyclictest3056-21PK-Backend12:00:102
162423799210,19cyclictest3056-21PK-Backend12:00:102
162423799210,19cyclictest3056-21PK-Backend11:25:292
1624232992119,1cyclictest3056-21PK-Backend08:32:061
1624232992119,1cyclictest3056-21PK-Backend07:19:001
1624232992119,1cyclictest0-21swapper/111:48:301
1624232992119,1cyclictest0-21swapper/111:25:521
1624232992119,1cyclictest0-21swapper/110:50:441
1624232992119,1cyclictest0-21swapper/109:43:321
1624232992119,1cyclictest0-21swapper/109:13:441
1624232992118,2cyclictest3056-21PK-Backend10:18:131
1624232992118,2cyclictest0-21swapper/109:05:121
1624232992110,9cyclictest1713515-21unin-run11:20:001
1624232992110,9cyclictest1713515-21unin-run11:20:001
162423299210,3cyclictest1738329-21python312:25:211
162423299210,20cyclictest3056-21PK-Backend10:41:511
162423299210,20cyclictest0-21swapper/112:04:051
162423299210,20cyclictest0-21swapper/112:04:041
162423299210,20cyclictest0-21swapper/111:09:431
162423299210,20cyclictest0-21swapper/107:59:021
1624229992119,1cyclictest0-21swapper/012:38:350
1624229992119,1cyclictest0-21swapper/012:31:040
1624229992119,1cyclictest0-21swapper/012:12:530
1624229992119,1cyclictest0-21swapper/011:52:250
1624229992119,1cyclictest0-21swapper/011:24:130
1624229992119,1cyclictest0-21swapper/010:18:010
1624229992119,1cyclictest0-21swapper/009:19:200
1624229992119,1cyclictest0-21swapper/009:03:260
1624229992119,1cyclictest0-21swapper/008:38:040
1624229992119,1cyclictest0-21swapper/008:07:080
1624229992119,1cyclictest0-21swapper/007:24:570
1624229992112,7cyclictest1728945-21ntp_states12:00:180
1624229992112,7cyclictest1728945-21ntp_states12:00:180
162422999210,20cyclictest0-21swapper/009:27:250
162422999210,20cyclictest0-21swapper/008:23:510
162422999210,20cyclictest0-21swapper/007:50:180
139099210,17rtkit-daemon3056-21PK-Backend12:26:593
16775882200,2sleep00-21swapper/009:40:010
16424902200,3sleep00-21swapper/008:00:130
1624245992018,1cyclictest3056-21PK-Backend11:17:163
1624245992018,1cyclictest3056-21PK-Backend11:17:163
1624245992018,1cyclictest3056-21PK-Backend08:52:443
1624245992018,1cyclictest3056-21PK-Backend08:46:353
1624245992018,1cyclictest0-21swapper/310:50:463
1624245992018,1cyclictest0-21swapper/308:24:253
1624245992018,1cyclictest0-21swapper/307:42:553
1624245992018,1cyclictest0-21swapper/307:31:533
1624245992013,6cyclictest1624888-21cat07:10:163
1624245992013,6cyclictest1624888-21cat07:10:163
162424599200,1cyclictest3056-21PK-Backend08:58:533
162424599200,1cyclictest3056-21PK-Backend08:11:013
162424599200,19cyclictest3056-21PK-Backend12:06:233
162424599200,19cyclictest0-21swapper/308:33:323
162423799203,16cyclictest1095-21runrttasks07:31:242
1624237992018,1cyclictest0-21swapper/211:59:082
1624237992018,1cyclictest0-21swapper/211:07:172
1624237992018,1cyclictest0-21swapper/210:31:472
1624237992018,1cyclictest0-21swapper/210:25:252
1624237992018,1cyclictest0-21swapper/209:27:592
1624237992018,1cyclictest0-21swapper/209:23:342
1624237992012,6cyclictest1699599-21df10:40:132
162423799200,1cyclictest0-21swapper/212:18:422
162423799200,19cyclictest0-21swapper/210:22:222
162423799200,19cyclictest0-21swapper/208:10:412
162423799200,19cyclictest0-21swapper/207:53:032
1624232992018,1cyclictest0-21swapper/111:56:071
1624232992018,1cyclictest0-21swapper/111:51:481
1624232992018,1cyclictest0-21swapper/111:37:491
1624232992018,1cyclictest0-21swapper/111:34:301
1624232992018,1cyclictest0-21swapper/110:48:281
1624232992018,1cyclictest0-21swapper/110:31:451
1624232992018,1cyclictest0-21swapper/109:35:401
1624232992018,1cyclictest0-21swapper/107:29:211
1624232992014,4cyclictest2156-21gnome-shell09:18:091
1624232992013,5cyclictest1643082-21pluginstate08:00:201
162423299200,1cyclictest0-21swapper/109:02:021
162423299200,19cyclictest0-21swapper/110:37:181
162423299200,19cyclictest0-21swapper/109:27:371
162423299200,19cyclictest0-21swapper/107:35:021
1624229992018,1cyclictest0-21swapper/012:15:570
1624229992018,1cyclictest0-21swapper/012:09:480
1624229992018,1cyclictest0-21swapper/011:57:450
1624229992018,1cyclictest0-21swapper/011:41:420
1624229992018,1cyclictest0-21swapper/011:26:290
1624229992018,1cyclictest0-21swapper/011:16:220
1624229992018,1cyclictest0-21swapper/011:16:210
1624229992018,1cyclictest0-21swapper/011:00:130
1624229992018,1cyclictest0-21swapper/010:54:520
1624229992018,1cyclictest0-21swapper/010:42:560
1624229992018,1cyclictest0-21swapper/010:00:380
1624229992018,1cyclictest0-21swapper/009:45:090
1624229992018,1cyclictest0-21swapper/009:22:590
1624229992018,1cyclictest0-21swapper/009:05:490
1624229992018,1cyclictest0-21swapper/007:56:340
1624229992018,1cyclictest0-21swapper/007:49:300
1624229992018,1cyclictest0-21swapper/007:14:500
1624229992018,1cyclictest0-21swapper/007:14:500
1624229992013,5cyclictest1723606-21sed11:45:200
162422999200,1cyclictest0-21swapper/007:42:080
162422999200,19cyclictest0-21swapper/010:46:130
162422999200,19cyclictest0-21swapper/009:57:200
162422999200,19cyclictest0-21swapper/009:40:430
162422999200,19cyclictest0-21swapper/008:47:090
162422999200,19cyclictest0-21swapper/008:41:150
162422999200,19cyclictest0-21swapper/008:13:370
139099200,16rtkit-daemon1624223-21cyclictest08:32:502
16408022190,4sleep20-21swapper/207:55:132
1624245991917,1cyclictest3056-21PK-Backend12:37:123
1624245991917,1cyclictest0-21swapper/308:35:553
1624245991917,1cyclictest0-21swapper/308:16:553
1624245991913,3cyclictest1733525-21latency_hist12:15:013
162424599191,16cyclictest1640976-21hddtemp_smartct07:55:163
162424599191,13cyclictest1095-21runrttasks07:35:093
162424599190,1cyclictest3056-21PK-Backend11:34:473
162424599190,1cyclictest3056-21PK-Backend11:25:073
162424599190,1cyclictest3056-21PK-Backend09:12:113
162423799197,9cyclictest883-21dbus-daemon10:45:222
162423799193,14cyclictest1735858-21tr12:20:142
1624237991917,1cyclictest0-21swapper/209:01:502
1624237991917,1cyclictest0-21swapper/208:38:192
1624237991917,1cyclictest0-21swapper/207:20:052
1624237991913,5cyclictest1688442-21cron10:10:002
162423799190,1cyclictest0-21swapper/212:34:102
162423799190,1cyclictest0-21swapper/211:50:562
162423799190,18cyclictest0-21swapper/211:16:362
162423799190,18cyclictest0-21swapper/211:16:362
1624232991917,1cyclictest0-21swapper/111:42:081
1624232991917,1cyclictest0-21swapper/111:21:331
1624232991917,1cyclictest0-21swapper/109:31:151
1624232991917,1cyclictest0-21swapper/108:55:051
1624232991914,3cyclictest1636862-21idleruntime-cro07:44:591
1624232991912,5cyclictest1659611-21sshd08:49:541
162423299190,1cyclictest3056-21PK-Backend10:55:041
162423299190,1cyclictest0-21swapper/108:51:541
162423299190,1cyclictest0-21swapper/108:06:521
162423299190,1cyclictest0-21swapper/107:46:521
1624229991917,1cyclictest3056-21PK-Backend08:55:420
1624229991917,1cyclictest0-21swapper/011:37:230
1624229991917,1cyclictest0-21swapper/009:50:240
1624229991917,1cyclictest0-21swapper/009:31:040
1624229991917,1cyclictest0-21swapper/008:34:190
1624229991912,5cyclictest1697948-21cat10:35:150
162422999190,1cyclictest0-21swapper/012:24:560
162422999190,1cyclictest0-21swapper/010:23:570
162422999190,1cyclictest0-21swapper/007:39:030
162422999190,1cyclictest0-21swapper/007:27:340
162422999190,18cyclictest3056-21PK-Backend08:16:340
162422999190,18cyclictest0-21swapper/011:34:460
1624245991816,1cyclictest3056-21PK-Backend09:32:053
1624245991816,1cyclictest0-21swapper/311:05:273
162424599181,13cyclictest1734560-21update-manager12:15:193
1624237991816,1cyclictest0-21swapper/212:35:102
1624237991816,1cyclictest0-21swapper/210:38:242
1624237991816,1cyclictest0-21swapper/209:19:082
1624237991816,1cyclictest0-21swapper/209:08:472
1624237991816,1cyclictest0-21swapper/207:29:042
1624237991813,4cyclictest34-21ksoftirqd/209:40:012
162423799180,1cyclictest0-21swapper/207:49:322
162423299188,8cyclictest2330-21gdbus12:30:191
162423299186,2cyclictest141rcu_preempt07:14:091
162423299186,2cyclictest141rcu_preempt07:14:081
162423299183,13cyclictest1657157-21ntpq08:40:191
1624232991816,1cyclictest0-21swapper/112:11:151
1624232991816,1cyclictest0-21swapper/107:51:251
1624232991812,4cyclictest1689365-21needreboot10:10:151
1624232991811,5cyclictest2175-21JS10:27:481
1624232991810,6cyclictest1648508-21ntp_states08:15:171
1624229991816,1cyclictest3056-21PK-Backend11:05:470
1624229991816,1cyclictest0-21swapper/008:27:370
1624229991812,5cyclictest1660798-21cat08:50:210
1624229991811,5cyclictest1075-21whoopsie10:05:080
1624229991810,6cyclictest1668565-21munin-run09:15:000
1624229991810,6cyclictest1626867-21seq07:15:120
1624245991715,1cyclictest3056-21PK-Backend07:48:163
1624245991712,3cyclictest42-21ksoftirqd/307:20:093
162424599170,5cyclictest1729841-21gpgconf12:05:013
162424599170,5cyclictest1729841-21gpgconf12:05:003
1624237991715,1cyclictest0-21swapper/209:41:532
1624237991715,1cyclictest0-21swapper/208:46:372
1624237991712,3cyclictest1650970-21idleruntime-cro08:25:002
162423299175,2cyclictest141rcu_preempt09:57:171
1624232991711,5cyclictest1632914-21sed07:30:231
162422999179,6cyclictest1689437-21ntpq10:10:170
1624229991710,5cyclictest1695691-21sed10:30:010
1624245991614,1cyclictest3056-21PK-Backend09:27:393
162423799162,13cyclictest1657893-21cron08:45:002
1624237991614,1cyclictest0-21swapper/211:41:312
1624237991614,1cyclictest0-21swapper/211:36:102
1624237991611,4cyclictest1722882-21cat11:45:112
162422999162,13cyclictest1633309-21cron07:35:000
162423799158,5cyclictest436-20systemd-journal09:56:532
162423799158,5cyclictest1624853-21cat07:10:152
162423799158,5cyclictest1624853-21cat07:10:152
162424599148,5cyclictest1685403-21tr10:00:143
162422999138,3cyclictest1706080-21grep11:00:010
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional