You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-30 - 23:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Thu Apr 30, 2026 12:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
907967993835,2cyclictest1006416-21kworker/u8:2+flush-259:009:19:590
907972993732,3cyclictest252-21systemd-journal10:16:221
907981993531,3cyclictest1121332-21kworker/u8:1+events_unbound10:57:343
907977993531,2cyclictest1099749-21fdisk10:15:572
907967993531,3cyclictest1105832-21kworker/u8:2+events_unbound10:51:440
907981993431,2cyclictest892820-21kworker/u8:1+flush-259:007:15:013
907972993430,3cyclictest1147776-21kworker/u8:0+flush-259:011:35:561
907967993330,2cyclictest1017532-21apt-get09:00:110
907981993228,3cyclictest1094427-21kworker/u8:1+events_unbound10:16:253
907967993227,4cyclictest1006416-21kworker/u8:2+flush-259:009:05:220
907981993128,2cyclictest3055436-21kworker/u8:2+events_unbound07:20:143
907967993124,5cyclictest1210116-21diskstats12:05:120
907981992925,3cyclictest907905-21latency_hist07:10:003
907981992925,3cyclictest1022450-21kworker/u8:0+flush-259:010:05:343
907967992925,3cyclictest1006636-21apt-get08:45:120
907981992824,3cyclictest1022450-21kworker/u8:0+flush-259:009:50:163
907981992823,3cyclictest252-21systemd-journal11:10:453
907967992824,3cyclictest1048388-21apt-get09:30:100
907981992724,2cyclictest918292-21kworker/u8:3+events_unbound07:35:193
907981992724,2cyclictest892820-21kworker/u8:1+events_unbound08:10:203
907981992721,4cyclictest1132146-21latency_hist10:50:003
907977992720,6cyclictest1162366-21ntpq11:15:192
907972992722,4cyclictest1137582-21/usr/sbin/munin10:55:141
907967992723,3cyclictest1084151-21kworker/u8:3+events_unbound10:16:200
907981992620,4cyclictest1204508-21latency_hist12:00:003
907972992622,2cyclictest954554-21/usr/sbin/munin07:55:131
907972992620,4cyclictest1189288-21apt-get11:45:121
907967992622,3cyclictest1220055-21kworker/u8:3+flush-259:012:30:160
907981992521,3cyclictest1121332-21kworker/u8:1+events_unbound10:40:203
907972992519,4cyclictest1068902-21/usr/sbin/munin09:50:131
907981992420,3cyclictest252-21systemd-journal08:05:463
907981992420,3cyclictest1111019-21kworker/u8:4+flush-259:011:30:593
907981992420,3cyclictest1022450-21kworker/u8:0+flush-259:009:35:003
907972992419,4cyclictest1111019-21kworker/u8:4+flush-259:011:10:151
907967992421,2cyclictest1225386-21kworker/u8:2+events_unbound12:25:240
907967992421,2cyclictest1225386-21kworker/u8:2+events_unbound12:25:230
907967992420,3cyclictest954353-21kworker/u8:0+flush-259:008:20:200
907981992319,3cyclictest1111019-21kworker/u8:4+events_unbound10:35:453
907981992319,3cyclictest1006416-21kworker/u8:2+flush-259:009:35:203
907981992319,3cyclictest1006416-21kworker/u8:2+events_unbound09:57:453
90797799234,6cyclictest1203788-21/usr/sbin/munin11:55:222
907977992319,3cyclictest1111019-21kworker/u8:4+flush-259:011:25:242
907977992318,3cyclictest1106046-21/usr/sbin/munin10:25:202
907967992319,3cyclictest1100861-21apt-get10:20:100
907967992319,3cyclictest1022450-21kworker/u8:0+flush-259:009:20:210
907967992317,4cyclictest1189022-21latency_hist11:45:010
907981992219,2cyclictest1006416-21kworker/u8:2+flush-259:009:25:313
907981992218,3cyclictest1214824-21kworker/u8:1+flush-259:012:20:213
907981992218,3cyclictest1163294-21latency_hist11:20:003
907981992218,3cyclictest1147776-21kworker/u8:0+events_unbound12:03:463
907981992217,4cyclictest1068665-21latency_hist09:50:003
907977992217,4cyclictest1068927-21apt-get09:50:112
907972992217,3cyclictest1240849-21apt-get12:35:121
907972992214,5cyclictest509-21rs:main1
90797299221,19cyclictest1126529-21basename10:45:011
907967992218,3cyclictest1105832-21kworker/u8:2+flush-259:010:35:000
907981992118,2cyclictest892820-21kworker/u8:1+events_unbound08:35:143
907981992117,3cyclictest954353-21kworker/u8:0+events_unbound08:20:213
90797799215,5cyclictest1115341-21ntpq10:30:202
907977992116,4cyclictest1067813-21ntpq09:45:212
907977992116,3cyclictest949402-21/usr/sbin/munin07:50:142
907977992111,6cyclictest944435-21diskstats07:45:132
907972992116,3cyclictest934128-21diskstats07:35:121
907972992116,3cyclictest252-21systemd-journal10:01:451
907972992116,3cyclictest252-21systemd-journal10:01:441
907972992115,3cyclictest1043227-21/usr/sbin/munin09:25:141
907967992117,3cyclictest892820-21kworker/u8:1+events_unbound08:37:070
907967992117,3cyclictest1147776-21kworker/u8:0+flush-259:011:30:220
907967992117,3cyclictest1022450-21kworker/u8:0+flush-259:010:10:180
907967992114,5cyclictest1057531-21ntpq09:35:190
907981992017,2cyclictest1006416-21kworker/u8:2+flush-259:010:00:233
907981992017,2cyclictest1006416-21kworker/u8:2+flush-259:010:00:223
907981992016,3cyclictest1006416-21kworker/u8:2+events_unbound08:57:473
907977992016,3cyclictest980102-21kworker/u8:3+flush-259:008:55:002
907977992016,3cyclictest1006416-21kworker/u8:2+flush-259:009:05:012
907977992014,5cyclictest252-21systemd-journal08:30:002
907967992016,3cyclictest1214824-21kworker/u8:1+events_unbound12:20:220
907981991917,2cyclictest980102-21kworker/u8:3+events_unbound08:25:133
907981991916,2cyclictest1121332-21kworker/u8:1+flush-259:011:25:013
907981991915,3cyclictest892820-21kworker/u8:1+events_unbound07:58:473
907981991915,3cyclictest1006416-21kworker/u8:2+events_unbound09:10:463
907981991913,4cyclictest1137345-21latency_hist10:55:003
907972991912,5cyclictest1230292-21latency_hist12:25:001
90797299191,16cyclictest1-21systemd10:25:011
907967991916,2cyclictest892820-21kworker/u8:1+flush-259:008:00:250
907967991915,3cyclictest954353-21kworker/u8:0+events_unbound08:15:200
907967991915,3cyclictest1153155-21/usr/sbin/munin11:10:230
907967991914,4cyclictest252-21systemd-journal07:24:580
907981991816,2cyclictest1111019-21kworker/u8:4+flush-259:011:40:203
907981991815,2cyclictest1022405-21sh09:05:013
907981991814,3cyclictest892820-21kworker/u8:1+events_unbound07:41:473
907981991814,3cyclictest1147776-21kworker/u8:0+events_unbound11:36:473
907981991814,3cyclictest1111019-21kworker/u8:4+events_unbound11:05:163
907981991813,4cyclictest1147751-21latency_hist11:05:003
90797799184,4cyclictest1125611-21ldconfig10:40:212
907977991814,3cyclictest1006416-21kworker/u8:2+flush-259:009:05:162
907977991813,4cyclictest252-21systemd-journal08:36:562
907972991815,2cyclictest1168434-21cron11:25:011
907972991814,3cyclictest1173852-21apt-get11:30:111
907972991814,2cyclictest0-21swapper/110:15:011
907972991813,3cyclictest1058676-21apt-get09:40:121
907967991814,3cyclictest980102-21kworker/u8:3+events_unbound08:25:220
907967991812,4cyclictest1073800-21latency_hist09:55:000
907967991811,5cyclictest1198488-21ntpq11:50:210
907981991714,2cyclictest892820-21kworker/u8:1+events_unbound07:50:003
907981991714,2cyclictest1240609-21kworker/u8:2+events_unbound12:35:193
907981991713,3cyclictest980102-21kworker/u8:3+events_unbound08:48:473
907981991713,3cyclictest892820-21kworker/u8:1+flush-259:007:15:233
907981991712,3cyclictest1110993-21latency_hist10:30:013
90797799176,9cyclictest974969-21latency_hist08:15:002
90797799173,4cyclictest933666-21sshd07:34:482
90797799173,4cyclictest917879-21sshd07:15:452
907977991714,2cyclictest924708-21kthreadcore07:25:172
907972991714,2cyclictest1200021-21irqcore11:55:161
907972991714,2cyclictest1199352-21sh11:54:591
907972991714,1cyclictest0-21swapper/112:05:001
907972991713,3cyclictest980102-21kworker/u8:3+flush-259:009:00:261
907972991712,4cyclictest1111019-21kworker/u8:4+flush-259:010:35:151
907967991715,1cyclictest0-21swapper/009:45:000
907967991714,3cyclictest1121332-21kworker/u8:1+events_unbound11:45:230
907967991714,2cyclictest60350irq/132-enp0s29f1:rx-012:15:140
907967991714,2cyclictest60350irq/132-enp0s29f1:rx-011:58:230
907967991713,3cyclictest892820-21kworker/u8:1+events_unbound07:10:180
907967991713,3cyclictest1147776-21kworker/u8:0+flush-259:012:00:190
907967991711,4cyclictest1027971-21diskstats09:10:120
907981991614,2cyclictest954353-21kworker/u8:0+flush-259:008:15:233
907981991612,3cyclictest1022450-21kworker/u8:0+events_unbound10:30:133
90797799164,6cyclictest1179196-21diskstats11:35:152
90797799164,3cyclictest1043553-21taskset09:25:152
90797799162,4cyclictest1147763-21basename11:05:002
90797799161,8cyclictest1209696-21munin-run12:05:002
907977991613,2cyclictest2971-21cron07:40:012
907977991613,2cyclictest1022450-21kworker/u8:0+flush-259:010:25:002
907977991610,4cyclictest1052361-21ntp_kernel_err09:30:222
907972991614,1cyclictest0-21swapper/112:15:001
907972991612,3cyclictest892820-21kworker/u8:1+flush-259:007:25:161
907972991611,3cyclictest980349-21apt-get08:20:121
907972991611,3cyclictest509-21rs:main1
907972991610,4cyclictest252-21systemd-journal07:50:011
907972991610,4cyclictest1168687-21/usr/sbin/munin11:25:131
907972991610,4cyclictest1105832-21kworker/u8:2+flush-259:010:45:161
907967991613,2cyclictest918292-21kworker/u8:3+flush-259:007:55:230
907967991613,2cyclictest60350irq/132-enp0s29f1:rx-012:10:230
907967991613,2cyclictest60350irq/132-enp0s29f1:rx-010:49:010
907967991612,3cyclictest1147776-21kworker/u8:0+flush-259:011:35:000
907967991612,3cyclictest1121332-21kworker/u8:1+flush-259:010:40:200
907967991612,2cyclictest0-21swapper/009:25:200
907981991513,2cyclictest1225386-21kworker/u8:2+events_unbound12:25:143
907981991513,2cyclictest1225386-21kworker/u8:2+events_unbound12:25:133
907981991512,2cyclictest1235352-21kworker/u8:0+events_unbound12:30:203
907981991512,2cyclictest1111019-21kworker/u8:4+flush-259:011:25:113
907981991511,3cyclictest1147776-21kworker/u8:0+flush-259:011:55:003
907981991511,3cyclictest1105811-21turbostat10:25:003
90797799157,6cyclictest1199357-21munin-plugin-st11:55:002
90797799154,2cyclictest1127497-21fw_forwarded_lo10:45:142
90797799153,4cyclictest1214655-21sshd12:07:462
90797799153,3cyclictest2109693-21snmpd12:23:082
90797799153,3cyclictest1230710-21df_inode12:25:132
90797799153,3cyclictest1230710-21df_inode12:25:132
90797799153,2cyclictest939260-21diskstats07:40:142
90797799153,2cyclictest1215280-21diskstats12:10:142
907977991513,1cyclictest1224326-21ntpq12:15:202
907977991512,2cyclictest954353-21kworker/u8:0+flush-259:008:05:162
907977991512,2cyclictest1137353-21turbostat.cron10:55:012
907977991511,2cyclictest1038116-21apt09:20:152
907967991513,1cyclictest60350irq/132-enp0s29f1:rx-011:05:160
907967991513,1cyclictest60350irq/132-enp0s29f1:rx-010:06:580
907967991513,1cyclictest60350irq/132-enp0s29f1:rx-008:33:210
907967991512,2cyclictest60350irq/132-enp0s29f1:rx-007:45:000
907967991512,2cyclictest60350irq/132-enp0s29f1:rx-007:30:220
907967991512,2cyclictest60350irq/132-enp0s29f1:rx-007:25:120
907967991511,3cyclictest1111019-21kworker/u8:4+events_unbound11:20:120
907981991411,2cyclictest1147776-21kworker/u8:0+flush-259:012:10:253
90797799149,4cyclictest252-21systemd-journal08:35:012
90797799149,3cyclictest1177903-21ntpq11:30:212
90797799148,5cyclictest1189035-21idleruntime-cro11:44:592
90797799148,4cyclictest1089293-21turbostat.cron10:10:002
90797799148,4cyclictest1017260-21sh09:00:012
90797799145,8cyclictest1054553-21ps09:35:162
90797799145,6cyclictest1079155-21sed10:00:002
90797799145,2cyclictest918661-21df07:20:122
90797799145,1cyclictest1058829-21df_inode09:40:132
90797799144,8cyclictest1245727-21basename12:40:002
90797799144,6cyclictest1032578-21chrt09:13:372
90797799143,2cyclictest976218-21irqstats08:15:172
90797799143,2cyclictest912192-21ntpq07:10:162
90797799143,2cyclictest1163832-21fw_conntrack11:20:122
90797799143,2cyclictest1141430-21memory10:55:202
907977991412,1cyclictest0-21swapper/211:10:012
90797299148,4cyclictest1136390-21ntpq10:50:191
907972991410,3cyclictest1068245-21unixbench_singl09:45:251
90796799149,4cyclictest918292-21kworker/u8:3+events_unbound07:35:130
907967991412,1cyclictest60350irq/132-enp0s29f1:rx-011:00:110
907967991412,1cyclictest60350irq/132-enp0s29f1:rx-010:35:090
907967991412,1cyclictest60350irq/132-enp0s29f1:rx-009:55:160
907967991412,1cyclictest60350irq/132-enp0s29f1:rx-009:45:190
907967991412,1cyclictest60350irq/132-enp0s29f1:rx-007:15:130
907967991411,2cyclictest60350irq/132-enp0s29f1:rx-007:45:230
907967991411,2cyclictest1006416-21kworker/u8:2+flush-259:008:55:210
907967991411,1cyclictest954313-21turbostat07:55:000
907967991410,3cyclictest1079674-21sed10:00:170
907967991410,3cyclictest1079674-21sed10:00:160
90798199139,3cyclictest1036947-21ntp_kernel_pll_09:15:203
90798199139,3cyclictest1022450-21kworker/u8:0+flush-259:010:12:523
90798199138,4cyclictest3055436-21kworker/u8:2+events_unbound07:29:593
90798199138,4cyclictest1027546-21latency_hist09:10:003
90798199136,5cyclictest994695-21ntpq08:30:223
907981991311,1cyclictest1022450-21kworker/u8:0+events_unbound09:40:223
90797799137,4cyclictest1194205-21basename11:50:002
90797799137,4cyclictest1006833-21tune2fs08:45:142
90797799136,5cyclictest1235969-21grep12:30:132
90797799135,6cyclictest1116634-21grep10:35:122
90797799135,6cyclictest1001300-21gpgv08:40:012
90797799133,2cyclictest1153339-21diskstats11:10:132
90797799133,2cyclictest1089852-21hddtemp_smartct10:10:172
907977991310,2cyclictest963637-21needreboot08:00:202
90797299139,3cyclictest990652-21apt-get08:30:111
90797299139,3cyclictest1111019-21kworker/u8:4+flush-259:011:05:171
90797299137,4cyclictest252-21systemd-journal12:35:001
90797299131,9cyclictest1012148-21awk08:55:001
907967991311,1cyclictest60350irq/132-enp0s29f1:rx-012:35:140
907967991311,1cyclictest60350irq/132-enp0s29f1:rx-011:28:320
907967991311,1cyclictest60350irq/132-enp0s29f1:rx-010:55:240
907967991311,1cyclictest60350irq/132-enp0s29f1:rx-008:10:110
907967991311,1cyclictest60350irq/132-enp0s29f1:rx-008:05:190
90798199129,2cyclictest892820-21kworker/u8:1+flush-259:008:00:153
90798199129,2cyclictest1038375-21fschecks_time09:20:173
90798199128,3cyclictest1005479-21ntp_kernel_pll_08:40:193
90798199126,4cyclictest929235-21grep07:30:163
907981991210,1cyclictest0-21swapper/312:10:003
90797799127,3cyclictest980460-21grep08:20:132
90797799127,3cyclictest954686-21tr07:55:112
90797799127,3cyclictest907980-21sed07:10:002
90797799125,5cyclictest1080393-21cut10:00:182
90797799125,5cyclictest1080393-21cut10:00:172
90797299129,1cyclictest0-21swapper/112:05:201
90797299128,3cyclictest892820-21kworker/u8:1+flush-259:008:07:161
90797299125,5cyclictest513-21gdbus09:34:591
90798199117,3cyclictest1147776-21kworker/u8:0+flush-259:011:46:013
90797299119,1cyclictest907963-21cyclictest12:15:141
90797299118,2cyclictest0-21swapper/107:35:001
90797299117,3cyclictest1006416-21kworker/u8:2+flush-259:009:05:511
90797299114,5cyclictest513-21gdbus08:35:511
90797299111,8cyclictest1235409-21awk12:30:001
90797299111,8cyclictest1235409-21awk12:30:001
90796799117,3cyclictest1012140-21cron08:55:010
90798199105,4cyclictest980102-21kworker/u8:3+flush-259:008:51:213
90797299107,2cyclictest1012611-21fschecks_count08:55:121
90797299106,3cyclictest954353-21kworker/u8:0+flush-259:008:03:161
90797299106,3cyclictest0-21swapper/110:05:311
90797299105,3cyclictest252-21systemd-journal10:34:451
90797299101,5cyclictest974008-21grep08:10:181
9079729996,2cyclictest949494-21cpuspeed_turbos07:50:121
9079729984,2cyclictest907905-21latency_hist07:10:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional