You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-23 - 22:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Thu Jul 23, 2026 12:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1431844995838,15cyclictest335557-21systemd-journal09:15:211
1431840995446,5cyclictest1452487-21latency_hist07:30:010
1431850994741,3cyclictest335557-21systemd-journal10:35:163
1431850994741,3cyclictest335557-21systemd-journal10:35:163
1431850994642,3cyclictest1598742-21kworker/u8:2+flush-259:011:10:263
1431850994539,4cyclictest1473546-21latency_hist07:50:013
1431846994441,2cyclictest1598742-21kworker/u8:2+events_unbound12:30:302
1431846994340,2cyclictest1572972-21kworker/u8:3+events_unbound09:53:252
1431846994340,2cyclictest1468414-21kworker/u8:2+events_unbound08:00:042
1431850994236,4cyclictest1530353-21gpgv08:45:013
1431844994237,3cyclictest1520212-21apt-get08:35:131
1431850994138,2cyclictest1598742-21kworker/u8:2+events_unbound11:02:263
1431850994138,2cyclictest1551876-21kworker/u8:0+events_unbound09:45:283
1431846994138,2cyclictest1551876-21kworker/u8:0+events_unbound09:37:262
1431850994037,2cyclictest1717913-21kworker/u8:0+flush-259:012:10:253
1431850994034,2cyclictest1388847-21kworker/u8:0+events_unbound07:25:203
1431850993936,2cyclictest1598742-21kworker/u8:2+events_unbound10:05:213
1431850993936,2cyclictest1520173-21kworker/u8:4+flush-259:008:45:263
1431846993936,2cyclictest1572972-21kworker/u8:3+events_unbound10:06:262
1431846993936,2cyclictest1520173-21kworker/u8:4+events_unbound08:47:262
1431846993936,2cyclictest1468414-21kworker/u8:2+events_unbound08:15:272
1431850993831,5cyclictest1727469-21ntpq11:50:223
1431846993832,4cyclictest335557-21systemd-journal07:54:592
1431846993734,2cyclictest1749283-21kworker/u8:4+events_unbound12:38:262
1431850993633,2cyclictest1717913-21kworker/u8:0+flush-259:011:45:463
1431850993633,2cyclictest1447240-21kworker/u8:1+events_unbound07:30:263
1431846993532,2cyclictest1598742-21kworker/u8:2+events_unbound11:25:272
1431850993431,2cyclictest1598742-21kworker/u8:2+events_unbound12:25:203
1431850993428,2cyclictest1388847-21kworker/u8:0+events_unbound07:20:123
1431844993429,3cyclictest1462840-21apt-get07:40:011
1431840993428,4cyclictest335557-21systemd-journal10:10:130
1431850993330,2cyclictest1468414-21kworker/u8:2+events_unbound08:15:143
1431850993329,3cyclictest1598742-21kworker/u8:2+events_unbound11:55:133
1431846993330,2cyclictest1520173-21kworker/u8:4+events_unbound09:04:102
1431846993328,4cyclictest1457658-21latency_hist07:35:012
1431840993329,2cyclictest1551876-21kworker/u8:0+flush-259:009:25:240
1431840993326,5cyclictest1441267-21ntpq07:15:210
1431850993229,2cyclictest1598742-21kworker/u8:2+events_unbound10:55:213
1431850993229,2cyclictest1388847-21kworker/u8:0+events_unbound08:28:273
1431850993227,2cyclictest1697039-21kworker/u8:1+events_unbound11:33:253
1431850993227,2cyclictest1572972-21kworker/u8:3+events_unbound11:22:263
1431850993128,2cyclictest1551876-21kworker/u8:0+events_unbound09:35:103
1431850993128,2cyclictest1377989-21kworker/u8:4+events_unbound07:14:553
1431850993127,3cyclictest1520173-21kworker/u8:4+events_unbound09:00:023
1431846993128,2cyclictest1598742-21kworker/u8:2+events_unbound10:56:072
1431846993128,2cyclictest1504240-21kworker/u8:1+events_unbound08:21:262
1431846993128,2cyclictest1447240-21kworker/u8:1+events_unbound07:44:592
1431840993124,5cyclictest335557-21systemd-journal08:25:090
1431850993027,2cyclictest1598742-21kworker/u8:2+events_unbound10:16:263
1431850993025,3cyclictest1535942-21kworker/u8:1+events_unbound09:14:263
1431846993028,2cyclictest1717913-21kworker/u8:0+events_unbound12:11:292
1431844993026,3cyclictest1551876-21kworker/u8:0+flush-259:009:10:211
1431844993024,4cyclictest1743921-21latency_hist12:10:001
1431844993023,5cyclictest335557-21systemd-journal09:30:091
1431840993024,4cyclictest1463349-21cat07:40:010
1431840993023,5cyclictest1562361-21base6409:15:010
1431850992926,2cyclictest1598742-21kworker/u8:2+events_unbound10:45:143
1431850992926,2cyclictest1551876-21kworker/u8:0+flush-259:009:50:233
1431850992926,2cyclictest1520173-21kworker/u8:4+events_unbound08:58:433
1431850992926,2cyclictest1388847-21kworker/u8:0+events_unbound08:13:263
1431850992926,2cyclictest1388847-21kworker/u8:0+events_unbound08:13:263
1431850992925,3cyclictest1551876-21kworker/u8:0+events_unbound09:25:103
1431850992925,3cyclictest1388847-21kworker/u8:0+flush-259:007:39:403
1431850992925,3cyclictest1377989-21kworker/u8:4+events_unbound07:15:313
1431846992927,2cyclictest1697039-21kworker/u8:1+events_unbound11:33:152
1431846992927,2cyclictest1640296-21kworker/u8:1+events_unbound10:32:052
1431846992926,2cyclictest1712657-21kworker/u8:3+events_unbound11:45:292
1431846992926,2cyclictest1535942-21kworker/u8:1+events_unbound09:12:222
1431844992922,5cyclictest1530430-21cp08:45:011
1431844992918,9cyclictest335557-21systemd-journal08:48:261
1431850992825,2cyclictest1509660-21kworker/u8:3+events_unbound08:52:083
1431850992824,3cyclictest1717913-21kworker/u8:0+events_unbound12:04:263
1431846992826,2cyclictest1717913-21kworker/u8:0+events_unbound12:06:272
1431846992825,2cyclictest1509660-21kworker/u8:3+events_unbound08:55:302
1431850992724,2cyclictest1572972-21kworker/u8:3+events_unbound10:00:013
1431850992724,2cyclictest1520173-21kworker/u8:4+events_unbound09:07:283
1431850992723,3cyclictest1567306-21kworker/u8:2+events_unbound09:23:253
1431850992721,4cyclictest1633999-21ntpq10:20:203
1431846992724,2cyclictest1717913-21kworker/u8:0+events_unbound12:04:252
1431844992723,3cyclictest1749283-21kworker/u8:4+flush-259:012:35:141
1431844992721,3cyclictest1551876-21kworker/u8:0+flush-259:009:45:001
1431850992623,2cyclictest1717913-21kworker/u8:0+events_unbound12:05:243
1431850992623,2cyclictest1504240-21kworker/u8:1+flush-259:008:20:293
1431850992623,2cyclictest1388847-21kworker/u8:0+events_unbound07:51:313
1431846992623,2cyclictest1388847-21kworker/u8:0+events_unbound07:29:552
1431846992621,4cyclictest1681509-21latency_hist11:10:012
1431844992619,5cyclictest335557-21systemd-journal07:24:591
1431840992623,2cyclictest1551876-21kworker/u8:0+flush-259:009:30:230
1431850992522,2cyclictest1598742-21kworker/u8:2+events_unbound12:35:263
1431850992522,2cyclictest1598742-21kworker/u8:2+events_unbound10:10:483
1431850992522,2cyclictest1535942-21kworker/u8:1+events_unbound09:15:073
1431850992518,2cyclictest1634930-21kworker/u8:0+events_unbound10:40:203
1431844992519,5cyclictest1650684-21apt-get10:40:121
1431840992521,3cyclictest1572972-21kworker/u8:3+flush-259:010:15:270
1431850992421,2cyclictest1749283-21kworker/u8:4+events_unbound12:24:253
1431850992421,2cyclictest1749069-21kworker/u8:1+events_unbound12:15:513
1431846992421,2cyclictest1551876-21kworker/u8:0+events_unbound09:56:032
1431844992419,3cyclictest1712820-21cron11:40:001
1431840992419,3cyclictest1739015-21apt-get12:05:130
1431840992417,6cyclictest1707674-21idleruntime-cro11:35:000
1431850992320,2cyclictest1598742-21kworker/u8:2+flush-259:011:35:173
1431850992320,2cyclictest1572972-21kworker/u8:3+events_unbound09:42:243
1431850992320,2cyclictest1388847-21kworker/u8:0+events_unbound08:06:483
1431846992320,2cyclictest1535942-21kworker/u8:1+events_unbound09:08:252
1431846992319,3cyclictest335557-21systemd-journal08:09:592
1431844992316,5cyclictest1665081-21ntpq10:50:201
1431840992319,3cyclictest1598742-21kworker/u8:2+flush-259:011:55:230
1431840992319,3cyclictest1489266-21apt-get08:05:120
1431840992318,3cyclictest1388847-21kworker/u8:0+flush-259:008:00:140
1431840992317,4cyclictest1447240-21kworker/u8:1+flush-259:007:45:170
1431850992219,2cyclictest1697039-21kworker/u8:1+events_unbound11:42:333
1431846992220,2cyclictest1609692-21kworker/u8:1+events_unbound10:03:262
1431846992219,2cyclictest1572972-21kworker/u8:3+events_unbound10:27:222
1431844992213,6cyclictest335557-21systemd-journal11:05:001
1431850992118,2cyclictest1572972-21kworker/u8:3+flush-259:011:15:213
1431846992117,3cyclictest335557-21systemd-journal08:44:592
1431844992117,3cyclictest335557-21systemd-journal07:49:461
1431844992115,4cyclictest1437264-21apt-get07:15:141
1431844992115,4cyclictest1388847-21kworker/u8:0+flush-259:007:30:001
1431840992116,3cyclictest1770024-21apt-get12:35:090
1431840992116,3cyclictest1718234-21/usr/sbin/munin11:45:140
1431840992115,4cyclictest1640305-21/usr/sbin/munin10:30:140
1431850992017,2cyclictest1572972-21kworker/u8:3+events_unbound09:55:183
1431850992017,2cyclictest1468414-21kworker/u8:2+events_unbound07:55:303
1431850992017,2cyclictest1447240-21kworker/u8:1+events_unbound07:43:543
1431850992017,2cyclictest1388847-21kworker/u8:0+events_unbound08:36:193
1431846992014,4cyclictest1519937-21latency_hist08:35:012
143184499203,15cyclictest1660809-21awk10:50:001
1431844992016,3cyclictest1598742-21kworker/u8:2+flush-259:011:30:001
1431840992017,2cyclictest1717913-21kworker/u8:0+events_unbound12:10:190
1431840992014,4cyclictest335557-21systemd-journal11:15:260
1431850991916,2cyclictest1697039-21kworker/u8:1+events_unbound11:27:163
1431846991916,2cyclictest1598742-21kworker/u8:2+events_unbound10:16:522
1431846991916,2cyclictest1468414-21kworker/u8:2+events_unbound07:57:332
1431846991915,3cyclictest1568017-21apt-get09:20:102
1431846991914,4cyclictest1598715-21latency_hist09:50:002
1431844991914,3cyclictest1718261-21apt-get11:45:131
1431844991914,3cyclictest1588679-21apt-get09:40:111
1431844991913,4cyclictest335557-21systemd-journal07:37:261
1431844991913,4cyclictest1509642-21latency_hist08:25:011
1431840991917,1cyclictest1598985-21apt-get09:50:120
1431840991916,2cyclictest1388847-21kworker/u8:0+events_unbound07:30:150
1431840991915,3cyclictest1634930-21kworker/u8:0+flush-259:010:40:250
1431840991913,4cyclictest1598742-21kworker/u8:2+flush-259:011:10:150
1431850991814,3cyclictest1749283-21kworker/u8:4+events_unbound12:33:553
143184699183,5cyclictest1469400-21sed07:45:182
1431846991815,2cyclictest1697039-21kworker/u8:1+events_unbound11:40:102
1431846991815,2cyclictest1634930-21kworker/u8:0+events_unbound10:54:572
1431846991815,2cyclictest1388847-21kworker/u8:0+events_unbound07:38:142
1431846991812,4cyclictest509-21rs:main2
1431844991815,2cyclictest1749283-21kworker/u8:4+flush-259:012:35:001
1431844991815,2cyclictest1614202-21sh10:04:591
1431844991814,3cyclictest1634930-21kworker/u8:0+flush-259:010:30:211
1431844991814,3cyclictest1598742-21kworker/u8:2+flush-259:010:55:211
1431844991813,4cyclictest1598742-21kworker/u8:2+flush-259:010:20:151
1431844991813,4cyclictest1551876-21kworker/u8:0+flush-259:010:00:001
1431844991812,5cyclictest1676570-21apt-get11:05:131
1431844991812,4cyclictest1702791-21/usr/sbin/munin11:30:141
1431844991812,4cyclictest1546274-21apt-get09:00:011
1431840991814,3cyclictest1388847-21kworker/u8:0+events_unbound08:10:190
1431840991814,3cyclictest1388847-21kworker/u8:0+events_unbound08:10:180
1431840991813,3cyclictest1640069-21turbostat10:29:590
1431846991714,2cyclictest1651709-21kthreadcore10:40:182
1431846991714,2cyclictest1598742-21kworker/u8:2+events_unbound10:21:032
1431846991714,2cyclictest1572972-21kworker/u8:3+events_unbound09:43:272
1431846991714,2cyclictest1520173-21kworker/u8:4+events_unbound08:50:092
1431846991714,2cyclictest1468414-21kworker/u8:2+events_unbound08:13:502
1431846991714,2cyclictest1468414-21kworker/u8:2+events_unbound08:13:492
1431846991713,3cyclictest1388847-21kworker/u8:0+events_unbound07:19:452
1431846991711,4cyclictest1514819-21latency_hist08:30:002
1431844991714,2cyclictest1598742-21kworker/u8:2+events_unbound10:25:191
1431844991714,2cyclictest1510313-21irqcore08:25:161
1431844991712,3cyclictest509-21rs:main1
1431840991713,3cyclictest1598742-21kworker/u8:2+events_unbound10:20:140
1431840991713,2cyclictest60350irq/132-enp0s29f1:rx-008:20:150
1431840991713,2cyclictest1-21systemd11:00:000
1431850991613,1cyclictest0-21swapper/310:30:003
1431850991612,3cyclictest1388847-21kworker/u8:0+events_unbound08:34:323
1431846991615,1cyclictest1749283-21kworker/u8:4+events_unbound12:27:272
1431846991614,2cyclictest1598742-21kworker/u8:2+events_unbound10:36:582
1431846991614,2cyclictest1598742-21kworker/u8:2+events_unbound10:36:582
1431846991613,2cyclictest1712657-21kworker/u8:3+events_unbound11:52:382
1431846991613,2cyclictest1598742-21kworker/u8:2+events_unbound11:56:542
1431846991613,2cyclictest1598742-21kworker/u8:2+events_unbound11:18:002
1431846991613,2cyclictest1572972-21kworker/u8:3+events_unbound09:27:272
1431846991613,2cyclictest1509660-21kworker/u8:3+events_unbound08:39:412
1431846991612,3cyclictest1598742-21kworker/u8:2+events_unbound11:02:202
1431846991612,3cyclictest1598742-21kworker/u8:2+events_unbound10:11:322
143184499169,5cyclictest1519076-21ntpq08:30:221
1431844991612,2cyclictest1499305-21idleruntime-cro08:15:001
1431844991612,2cyclictest1499305-21idleruntime-cro08:14:591
1431844991611,4cyclictest1520173-21kworker/u8:4+flush-259:009:00:011
143184099169,5cyclictest1477769-21ntpq07:50:210
1431840991613,2cyclictest60350irq/132-enp0s29f1:rx-012:19:190
1431840991612,2cyclictest1733814-21apt-get12:00:110
1431840991612,2cyclictest1572957-21turbostat09:25:010
1431840991611,3cyclictest335557-21systemd-journal09:38:250
1431840991610,4cyclictest1432038-21/usr/sbin/munin07:10:150
1431850991512,2cyclictest1634930-21kworker/u8:0+events_unbound10:52:173
143184699152,3cyclictest41-21ksoftirqd/212:15:172
143184699151,4cyclictest41-21ksoftirqd/211:20:202
1431846991512,2cyclictest391rcuc/207:15:002
1431846991512,2cyclictest2587708-21kworker/u8:5+events_unbound07:24:182
1431846991512,2cyclictest1749283-21kworker/u8:4+events_unbound12:22:142
1431846991511,3cyclictest1598742-21kworker/u8:2+events_unbound11:12:072
1431846991511,3cyclictest1551876-21kworker/u8:0+events_unbound09:32:032
143184499159,4cyclictest1598742-21kworker/u8:2+flush-259:012:30:011
1431844991510,4cyclictest1749283-21kworker/u8:4+flush-259:012:25:001
1431840991513,2cyclictest60350irq/132-enp0s29f1:rx-007:36:200
1431840991513,1cyclictest60350irq/132-enp0s29f1:rx-011:50:310
1431840991513,1cyclictest60350irq/132-enp0s29f1:rx-011:25:230
1431840991513,1cyclictest60350irq/132-enp0s29f1:rx-011:20:190
1431840991513,1cyclictest60350irq/132-enp0s29f1:rx-010:00:240
1431840991512,2cyclictest60350irq/132-enp0s29f1:rx-012:27:210
1431840991511,2cyclictest1445326-21kthreadcore07:20:190
1431840991510,4cyclictest1749069-21kworker/u8:1+events_unbound12:20:130
1431850991410,3cyclictest1598742-21kworker/u8:2+events_unbound11:05:443
143184699147,2cyclictest41-21ksoftirqd/209:15:012
143184699144,3cyclictest41-21ksoftirqd/207:10:002
143184699140,4cyclictest1658832-21kthreadcore10:45:192
143184099149,4cyclictest1388847-21kworker/u8:0+events_unbound08:35:000
143184099147,5cyclictest1713267-21diskstats11:40:140
1431840991412,1cyclictest60350irq/132-enp0s29f1:rx-012:31:140
1431840991412,1cyclictest60350irq/132-enp0s29f1:rx-009:15:190
1431840991412,1cyclictest60350irq/132-enp0s29f1:rx-008:57:030
1431840991411,2cyclictest60350irq/132-enp0s29f1:rx-011:35:140
1431840991411,2cyclictest60350irq/132-enp0s29f1:rx-010:50:120
1431840991411,2cyclictest60350irq/132-enp0s29f1:rx-010:36:400
1431840991411,2cyclictest60350irq/132-enp0s29f1:rx-010:36:390
1431840991411,2cyclictest60350irq/132-enp0s29f1:rx-008:45:240
1431840991411,2cyclictest60350irq/132-enp0s29f1:rx-008:40:120
1431840991411,2cyclictest60350irq/132-enp0s29f1:rx-008:37:020
143184499139,3cyclictest1468414-21kworker/u8:2+flush-259:008:05:361
1431840991311,1cyclictest60350irq/132-enp0s29f1:rx-010:05:120
1431840991311,1cyclictest60350irq/132-enp0s29f1:rx-009:45:140
1431840991311,1cyclictest60350irq/132-enp0s29f1:rx-008:15:140
1431840991310,2cyclictest60350irq/132-enp0s29f1:rx-009:55:210
1431840991310,2cyclictest1554410-21sort09:05:180
1431840991310,2cyclictest1482880-21needreboot07:55:210
143185099129,2cyclictest1640296-21kworker/u8:1+events_unbound10:32:283
143184499128,3cyclictest1723613-21cut11:50:151
143184499126,4cyclictest1749292-21/usr/sbin/munin12:15:141
143184499126,4cyclictest1744189-21apt-get12:10:121
1431844991210,1cyclictest0-21swapper/111:20:001
143184099129,2cyclictest60350irq/132-enp0s29f1:rx-011:00:210
1431840991211,1cyclictest61150irq/133-enp0s29f1:tx-010:45:170
1431840991211,1cyclictest60350irq/132-enp0s29f1:rx-011:05:180
1431840991211,1cyclictest60350irq/132-enp0s29f1:rx-009:40:030
1431840991210,2cyclictest60350irq/132-enp0s29f1:rx-009:01:560
143184499117,3cyclictest335557-21systemd-journal10:17:001
143184499116,3cyclictest1603877-21cron09:55:001
143184499107,2cyclictest1455052-21tr07:30:171
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional