You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-26 - 18:11

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Sun Apr 26, 2026 12:44:07)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
824163995754,2cyclictest1053859-21kworker/u8:2+flush-259:011:15:230
824167994946,2cyclictest938949-21kworker/u8:0+flush-259:009:20:001
824172994744,2cyclictest943749-21fwupd09:01:202
824163994542,2cyclictest1053859-21kworker/u8:2+events_unbound11:05:230
824167994439,3cyclictest1053859-21kworker/u8:2+flush-259:010:50:141
824163994239,2cyclictest840047-21kworker/u8:0+flush-259:007:50:240
824167994137,3cyclictest1059677-21kworker/u8:0+flush-259:011:04:591
824163994138,2cyclictest818794-21kworker/u8:2+flush-259:007:35:210
824163994137,3cyclictest1053859-21kworker/u8:2+flush-259:011:40:210
824163994037,2cyclictest1002205-21kworker/u8:3+flush-259:010:30:150
824163993936,2cyclictest818794-21kworker/u8:2+flush-259:007:55:160
824163993936,2cyclictest1022841-21kworker/u8:0+flush-259:010:35:240
824163993935,3cyclictest1028201-21kworker/u8:1+events_unbound10:45:170
824163993834,3cyclictest887727-21kworker/u8:5+flush-259:008:30:260
824163993833,4cyclictest839815-21latency_hist07:25:010
824163993831,5cyclictest1063684-21ntpq10:55:220
824167993732,3cyclictest1148662-21apt-get12:20:121
824167993732,3cyclictest1002205-21kworker/u8:3+flush-259:010:20:141
824163993734,2cyclictest211rcuc/009:35:010
824172993630,4cyclictest860957-21gpgv07:45:022
824167993633,2cyclictest1053859-21kworker/u8:2+events_unbound11:35:211
824167993528,5cyclictest1137790-21cp12:10:001
824163993532,2cyclictest876758-21kworker/u8:3+flush-259:009:40:240
824167993429,4cyclictest960231-21/usr/sbin/munin09:20:161
824167993429,3cyclictest887727-21kworker/u8:5+flush-259:009:00:221
824167993429,3cyclictest1122049-21/usr/sbin/munin11:55:151
824167993426,5cyclictest850291-21cat07:35:011
824163993430,3cyclictest1028201-21kworker/u8:1+flush-259:011:10:230
824163993430,3cyclictest1028201-21kworker/u8:1+flush-259:011:10:220
824163993428,5cyclictest1105045-21ntpq11:35:210
824175993327,4cyclictest1153549-21latency_hist12:25:013
824172993328,4cyclictest840047-21kworker/u8:0+flush-259:007:50:172
824167993330,2cyclictest1028201-21kworker/u8:1+events_unbound11:45:021
824167993328,3cyclictest1116688-21kworker/u8:3+flush-259:012:00:261
824163993331,1cyclictest887727-21kworker/u8:5+events_unbound09:00:110
824163993330,2cyclictest938949-21kworker/u8:0+flush-259:009:15:240
824163993329,3cyclictest1059677-21kworker/u8:0+events_unbound11:00:160
824163993329,3cyclictest1028201-21kworker/u8:1+flush-259:012:35:150
824172993228,3cyclictest882517-21kworker/u8:4+flush-259:008:10:202
824167993227,3cyclictest928917-21apt-get08:50:141
824167993227,3cyclictest928917-21apt-get08:50:131
824163993228,3cyclictest882517-21kworker/u8:4+flush-259:008:15:150
824175993125,4cyclictest1064598-21latency_hist11:00:003
824172993127,3cyclictest986270-21kworker/u8:2+flush-259:010:10:212
824167993127,3cyclictest1002205-21kworker/u8:3+flush-259:010:05:261
824167993126,4cyclictest1106189-21apt-get11:40:101
824163993128,2cyclictest1053859-21kworker/u8:2+flush-259:011:50:170
824163993128,2cyclictest1053859-21kworker/u8:2+flush-259:011:20:240
824163993128,2cyclictest1028201-21kworker/u8:1+events_unbound11:25:180
824172993023,4cyclictest1007333-21latency_hist10:05:022
824167993027,2cyclictest882517-21kworker/u8:4+events_unbound08:30:151
824167993026,3cyclictest818794-21kworker/u8:2+events_unbound07:10:131
824167993024,4cyclictest838890-21ntpq07:20:211
824163993027,2cyclictest938949-21kworker/u8:0+events_unbound09:10:220
824163993026,3cyclictest1153822-21apt-get12:25:120
824172992925,3cyclictest1028201-21kworker/u8:1+flush-259:012:15:222
824172992922,5cyclictest908476-21diskstats08:30:142
824167992926,2cyclictest887727-21kworker/u8:5+events_unbound08:20:251
824167992926,2cyclictest818794-21kworker/u8:2+events_unbound07:40:151
824167992926,2cyclictest1002205-21kworker/u8:3+flush-259:010:40:151
824163992925,3cyclictest996695-21kworker/u8:0+flush-259:009:55:190
824175992824,3cyclictest850144-21gpgv07:35:013
824167992825,2cyclictest882517-21kworker/u8:4+events_unbound08:15:181
824167992825,2cyclictest819005-21kworker/u8:3+events_unbound07:10:011
824167992824,3cyclictest949256-21kworker/u8:1+flush-259:009:10:171
824167992822,4cyclictest252-21systemd-journal09:49:341
824167992822,4cyclictest1080333-21/usr/sbin/munin11:15:151
824163992826,2cyclictest819005-21kworker/u8:3+flush-259:007:34:130
824163992825,2cyclictest1053859-21kworker/u8:2+events_unbound11:55:190
824175992724,2cyclictest1111174-21gpgv11:45:013
824175992723,3cyclictest938949-21kworker/u8:0+flush-259:009:25:173
824172992721,4cyclictest923775-21/usr/sbin/munin08:45:242
824172992721,4cyclictest923775-21/usr/sbin/munin08:45:232
824172992721,4cyclictest866400-21apt-get07:50:002
824172992720,5cyclictest902056-21ntpq08:20:212
82417299271,25cyclictest1-21systemd09:20:002
824167992724,2cyclictest882517-21kworker/u8:4+flush-259:008:10:151
824167992724,2cyclictest1002205-21kworker/u8:3+flush-259:010:25:251
824167992723,3cyclictest938949-21kworker/u8:0+events_unbound09:25:231
824167992723,3cyclictest1090681-21apt-get11:25:111
824163992724,2cyclictest887727-21kworker/u8:5+flush-259:008:35:150
824163992724,2cyclictest887727-21kworker/u8:5+events_unbound08:20:160
824163992724,2cyclictest818794-21kworker/u8:2+flush-259:008:00:240
824163992722,4cyclictest1059445-21turbostat10:54:590
824163992722,3cyclictest1100731-21latency_hist11:35:010
824175992620,4cyclictest509-21rs:main3
824175992619,5cyclictest818794-21kworker/u8:2+flush-259:008:05:183
824167992621,4cyclictest985318-21ntpq09:40:211
824163992623,2cyclictest986270-21kworker/u8:2+flush-259:010:10:200
824163992623,2cyclictest876758-21kworker/u8:3+flush-259:008:55:150
824163992623,2cyclictest819005-21kworker/u8:3+flush-259:007:15:000
824163992623,2cyclictest1028201-21kworker/u8:1+flush-259:012:30:150
824172992522,2cyclictest509-21rs:main2
824172992520,4cyclictest876758-21kworker/u8:3+flush-259:008:40:262
824172992520,4cyclictest1028201-21kworker/u8:1+flush-259:010:45:242
824167992523,1cyclictest840047-21kworker/u8:0+events_unbound07:50:161
824167992522,2cyclictest876758-21kworker/u8:3+events_unbound09:05:241
824167992522,2cyclictest840047-21kworker/u8:0+events_unbound07:45:131
824167992522,2cyclictest1028201-21kworker/u8:1+flush-259:012:35:251
824167992521,3cyclictest876758-21kworker/u8:3+events_unbound08:05:231
824167992521,3cyclictest252-21systemd-journal07:20:001
824167992521,3cyclictest1028201-21kworker/u8:1+flush-259:011:05:141
824167992520,4cyclictest882284-21latency_hist08:05:001
824163992523,2cyclictest1002205-21kworker/u8:3+flush-259:010:20:220
824163992522,2cyclictest938949-21kworker/u8:0+events_unbound09:25:170
824163992522,2cyclictest840047-21kworker/u8:0+flush-259:007:25:160
824175992420,3cyclictest840047-21kworker/u8:0+flush-259:007:50:013
824175992419,3cyclictest913475-21apt-get08:35:113
82417599240,22cyclictest1-21systemd12:00:003
824172992419,3cyclictest252-21systemd-journal10:31:332
824167992422,2cyclictest887727-21kworker/u8:5+events_unbound08:44:271
824167992421,2cyclictest986270-21kworker/u8:2+flush-259:010:10:211
824167992421,2cyclictest986270-21kworker/u8:2+flush-259:009:50:241
824167992421,2cyclictest1138382-21apt-get12:10:131
824167992420,3cyclictest1137717-21kworker/u8:0+events_unbound12:19:341
824163992419,4cyclictest887727-21kworker/u8:5+flush-259:008:55:010
824163992419,4cyclictest887727-21kworker/u8:5+flush-259:008:55:000
824175992317,4cyclictest252-21systemd-journal11:25:023
824172992319,3cyclictest1028201-21kworker/u8:1+flush-259:012:35:012
824172992318,3cyclictest252-21systemd-journal09:54:342
824167992320,3cyclictest840047-21kworker/u8:0+events_unbound07:35:331
824167992320,2cyclictest996695-21kworker/u8:0+events_unbound10:00:201
824167992320,2cyclictest876758-21kworker/u8:3+events_unbound07:59:181
824167992318,4cyclictest938949-21kworker/u8:0+events_unbound09:35:161
824163992320,2cyclictest986270-21kworker/u8:2+events_unbound10:15:210
824163992320,2cyclictest1002205-21kworker/u8:3+flush-259:010:05:210
824163992319,3cyclictest1116688-21kworker/u8:3+flush-259:012:05:160
824175992215,4cyclictest1017663-21latency_hist10:15:003
824172992217,3cyclictest840047-21kworker/u8:0+flush-259:007:30:162
824172992217,3cyclictest1053859-21kworker/u8:2+flush-259:011:19:592
824172992216,4cyclictest1028201-21kworker/u8:1+flush-259:010:25:172
824172992215,5cyclictest252-21systemd-journal07:12:332
824172992215,5cyclictest1027969-21latency_hist10:25:012
824167992219,2cyclictest1002205-21kworker/u8:3+flush-259:010:30:251
824167992217,4cyclictest997245-21apt-get09:55:111
824163992219,2cyclictest882517-21kworker/u8:4+flush-259:008:10:190
824163992219,2cyclictest840047-21kworker/u8:0+events_unbound07:45:210
824163992218,3cyclictest986918-21apt-get09:45:140
824163992218,3cyclictest1116688-21kworker/u8:3+events_unbound12:00:230
824175992115,4cyclictest954416-21latency_hist09:15:013
824175992115,4cyclictest252-21systemd-journal12:09:333
824172992118,2cyclictest0-21swapper/211:00:002
824172992115,4cyclictest252-21systemd-journal08:36:342
824172992115,4cyclictest1138355-21/usr/sbin/munin12:10:152
824167992118,2cyclictest986270-21kworker/u8:2+flush-259:010:15:231
824167992118,2cyclictest1053859-21kworker/u8:2+flush-259:011:30:251
824163992118,2cyclictest818794-21kworker/u8:2+events_unbound08:05:180
824175992013,5cyclictest1033122-21latency_hist10:30:003
824172992017,2cyclictest876758-21kworker/u8:3+events_unbound09:40:182
824172992016,3cyclictest1059677-21kworker/u8:0+flush-259:011:00:262
824172992015,4cyclictest887727-21kworker/u8:5+flush-259:009:00:012
824172992014,4cyclictest949256-21kworker/u8:1+flush-259:009:10:162
824172992013,5cyclictest1137684-21latency_hist12:10:012
824167992017,2cyclictest938949-21kworker/u8:0+events_unbound09:30:171
824167992017,2cyclictest887727-21kworker/u8:5+events_unbound08:45:201
824167992017,2cyclictest887727-21kworker/u8:5+events_unbound08:45:201
824167992017,2cyclictest1148413-21kworker/u8:2+events_unbound12:25:171
824167992017,2cyclictest1137717-21kworker/u8:0+flush-259:012:30:251
824167992014,4cyclictest252-21systemd-journal11:00:001
824163992017,2cyclictest986270-21kworker/u8:2+events_unbound09:50:190
824163992017,2cyclictest1053859-21kworker/u8:2+events_unbound11:45:240
824175991917,1cyclictest818794-21kworker/u8:2+events_unbound07:55:183
824175991915,3cyclictest986270-21kworker/u8:2+flush-259:010:10:013
824172991915,3cyclictest840047-21kworker/u8:0+flush-259:007:35:262
824172991915,3cyclictest1059677-21kworker/u8:0+events_unbound11:25:002
824172991915,3cyclictest1028201-21kworker/u8:1+flush-259:012:30:012
824167991916,2cyclictest876758-21kworker/u8:3+flush-259:008:55:211
824167991916,2cyclictest876758-21kworker/u8:3+events_unbound08:35:141
824167991916,1cyclictest0-21swapper/110:50:011
824163991917,2cyclictest819005-21kworker/u8:3+events_unbound07:15:230
824163991916,2cyclictest876758-21kworker/u8:3+events_unbound09:05:260
82417599182,5cyclictest1131393-21/usr/sbin/munin12:00:253
82417299182,10cyclictest877151-21munin-run08:00:002
824172991815,2cyclictest1053859-21kworker/u8:2+events_unbound11:30:222
824172991815,2cyclictest1022832-21munin-plugin-st10:20:002
824172991814,3cyclictest876758-21kworker/u8:3+flush-259:009:25:162
824172991813,3cyclictest876758-21kworker/u8:3+flush-259:009:05:172
824172991813,3cyclictest1002205-21kworker/u8:3+flush-259:010:05:192
824167991812,2cyclictest887727-21kworker/u8:5+events_unbound08:25:221
824163991816,2cyclictest887727-21kworker/u8:5+events_unbound08:25:260
824163991815,3cyclictest887727-21kworker/u8:5+events_unbound08:48:340
824163991815,3cyclictest887727-21kworker/u8:5+events_unbound08:48:330
824163991815,2cyclictest996695-21kworker/u8:0+events_unbound10:00:160
824163991814,3cyclictest1137717-21kworker/u8:0+events_unbound12:20:350
824163991814,3cyclictest1137717-21kworker/u8:0+events_unbound12:11:350
82417599172,4cyclictest1111910-21df_inode11:45:133
824175991712,3cyclictest0-21swapper/310:40:003
824175991710,5cyclictest1143245-21latency_hist12:15:003
824175991710,5cyclictest1007340-21awk10:05:003
824172991714,1cyclictest1137717-21kworker/u8:0+events_unbound12:20:222
824172991713,3cyclictest1053859-21kworker/u8:2+flush-259:011:40:252
824172991710,5cyclictest252-21systemd-journal07:23:332
82416799179,3cyclictest1022841-21kworker/u8:0+events_unbound10:35:181
824167991713,3cyclictest1053859-21kworker/u8:2+events_unbound11:20:021
824163991714,2cyclictest938949-21kworker/u8:0+flush-259:009:35:190
824163991712,4cyclictest1032274-21ntpq10:25:230
82417599169,5cyclictest986416-21cp09:45:003
82417599163,3cyclictest1021957-21ntpq10:15:223
824175991613,2cyclictest61850irq/147-enp0s29f1:tx-709:47:043
824175991613,2cyclictest0-21swapper/308:25:003
824175991610,5cyclictest1053811-21apt-get10:50:013
82417299169,5cyclictest252-21systemd-journal08:19:592
824172991611,4cyclictest996695-21kworker/u8:0+flush-259:009:55:212
824172991611,3cyclictest986226-21apt-get09:45:022
824172991611,3cyclictest1116688-21kworker/u8:3+flush-259:012:00:162
824163991614,1cyclictest60350irq/132-enp0s29f1:rx-009:20:260
82417599158,5cyclictest954928-21cat09:15:003
82417599158,5cyclictest938949-21kworker/u8:0+flush-259:009:35:173
82417599152,4cyclictest1101162-21sh11:35:143
824175991511,3cyclictest1053859-21kworker/u8:2+flush-259:011:30:193
824172991511,3cyclictest1053859-21kworker/u8:2+events_unbound10:50:192
824163991513,1cyclictest60350irq/132-enp0s29f1:rx-008:40:170
82417599149,3cyclictest0-21swapper/308:15:003
82417599148,4cyclictest996695-21kworker/u8:0+flush-259:009:55:163
82417599148,4cyclictest971006-21/usr/sbin/munin09:30:133
82417599143,6cyclictest912285-21ntpq08:30:203
82417599143,6cyclictest1162979-21ntpq12:30:203
82417599143,2cyclictest881369-21ntpq08:00:213
82417599141,3cyclictest867256-21/usr/sbin/munin07:50:153
824175991410,3cyclictest1080074-21idleruntime-cro11:15:003
824175991410,3cyclictest1080074-21idleruntime-cro11:14:593
824172991410,3cyclictest1137717-21kworker/u8:0+flush-259:012:35:172
824167991411,2cyclictest840047-21kworker/u8:0+events_unbound07:29:351
824163991412,1cyclictest60350irq/132-enp0s29f1:rx-012:15:150
824163991411,2cyclictest824135-21apt-key07:10:010
824163991410,3cyclictest840047-21kworker/u8:0+events_unbound07:41:030
82417599137,4cyclictest0-21swapper/309:52:043
82417599133,2cyclictest1065017-21/usr/sbin/munin11:00:153
82417599132,7cyclictest855436-21runrttasks07:37:183
82417599132,6cyclictest907237-21ntp_states08:25:243
82417599132,2cyclictest944958-21irqcore09:05:163
82417599132,2cyclictest929200-21fw_conntrack08:50:163
82417599132,2cyclictest929200-21fw_conntrack08:50:163
82417599131,7cyclictest939473-21fw_forwarded_lo09:00:133
82417599131,7cyclictest50-21ksoftirqd/310:30:163
82417599131,3cyclictest825370-21irqcore07:10:163
824175991312,1cyclictest491ktimers/308:45:223
824175991312,1cyclictest491ktimers/308:45:213
824175991310,1cyclictest0-21swapper/311:19:593
82417299139,3cyclictest938949-21kworker/u8:0+flush-259:009:20:182
82417299139,3cyclictest1022841-21kworker/u8:0+flush-259:010:40:252
82417299139,2cyclictest0-21swapper/208:30:002
82417299138,3cyclictest882553-21apt-get08:05:132
824167991310,2cyclictest1028201-21kworker/u8:1+events_unbound11:10:201
824167991310,2cyclictest1028201-21kworker/u8:1+events_unbound11:10:191
82417599129,2cyclictest1043923-21fschecks_count10:40:123
82417599121,6cyclictest918763-21df08:40:143
82417599121,2cyclictest859944-21ntpq07:40:223
82417599121,2cyclictest844083-21ntpq07:25:233
82417299126,3cyclictest975937-21awk09:35:002
82417299122,2cyclictest191rcu_preempt11:50:212
82416799128,3cyclictest1053859-21kworker/u8:2+events_unbound11:50:221
82417599119,1cyclictest0-21swapper/311:50:133
82417599117,2cyclictest0-21swapper/310:50:003
82417599111,6cyclictest50-21ksoftirqd/308:20:003
82417599111,2cyclictest50-21ksoftirqd/307:20:153
82417299118,2cyclictest840149-21cpuspeed_turbos07:25:152
82417299118,2cyclictest1091211-21irqcore11:25:172
82417299115,4cyclictest252-21systemd-journal10:39:592
82417599107,2cyclictest960361-21cpuspeed_turbos09:20:143
82417299106,3cyclictest252-21systemd-journal08:00:332
82417299106,2cyclictest932934-21ntpq08:50:242
82417299106,2cyclictest932934-21ntpq08:50:242
82417299106,2cyclictest0-21swapper/207:15:172
8241729996,2cyclictest1075457-21cut11:10:152
8241729996,2cyclictest1075457-21cut11:10:152
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional