You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-20 - 11:14

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack8slot6s.osadl.org (updated Thu Nov 20, 2025 00:43:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
22474993838,0cyclictest25065-21fwupd23:47:031
22482992018,2cyclictest749-21ntpd22:13:513
22474991919,0cyclictest66850irq/125-lan20:40:291
22474991918,1cyclictest66850irq/125-lan22:40:181
22474991918,1cyclictest66850irq/125-lan21:31:041
22474991918,1cyclictest66850irq/125-lan00:22:381
22474991818,0cyclictest66850irq/125-lan23:00:201
22474991818,0cyclictest66850irq/125-lan21:20:421
22474991818,0cyclictest66850irq/125-lan00:15:191
22474991818,0cyclictest66850irq/125-lan00:04:131
22474991817,1cyclictest66850irq/125-lan23:31:101
22474991817,1cyclictest66850irq/125-lan23:28:081
22474991817,1cyclictest66850irq/125-lan23:22:501
22474991817,1cyclictest66850irq/125-lan22:50:151
22474991817,1cyclictest66850irq/125-lan21:58:431
22474991817,1cyclictest66850irq/125-lan21:49:161
22474991817,1cyclictest66850irq/125-lan21:40:171
22474991817,1cyclictest66850irq/125-lan00:30:281
22474991817,1cyclictest66850irq/125-lan00:07:521
22474991717,0cyclictest66850irq/125-lan23:10:121
22474991717,0cyclictest66850irq/125-lan22:14:191
22474991717,0cyclictest66850irq/125-lan22:00:101
22474991717,0cyclictest66850irq/125-lan21:39:211
22474991717,0cyclictest66850irq/125-lan00:10:171
22474991716,1cyclictest66850irq/125-lan23:57:461
22474991716,1cyclictest66850irq/125-lan23:07:551
22474991716,1cyclictest66850irq/125-lan22:56:231
22474991716,1cyclictest66850irq/125-lan22:38:371
22474991716,1cyclictest66850irq/125-lan22:21:421
22474991716,1cyclictest66850irq/125-lan22:18:381
22474991716,1cyclictest66850irq/125-lan22:05:161
22474991716,1cyclictest66850irq/125-lan21:50:291
22474991715,1cyclictest27457-21kworker/1:0+events22:31:561
22474991616,0cyclictest66850irq/125-lan23:50:401
22474991616,0cyclictest66850irq/125-lan22:45:181
22474991616,0cyclictest66850irq/125-lan21:29:051
22474991616,0cyclictest66850irq/125-lan19:50:081
22474991616,0cyclictest66850irq/125-lan19:34:491
22474991616,0cyclictest66850irq/125-lan00:35:151
22474991616,0cyclictest66850irq/125-lan00:25:171
22474991615,1cyclictest66850irq/125-lan23:19:361
22474991615,1cyclictest66850irq/125-lan21:15:021
22474991615,1cyclictest66850irq/125-lan21:10:201
22474991615,1cyclictest66850irq/125-lan19:55:121
22482991515,0cyclictest771-21snmpd23:54:133
22482991515,0cyclictest771-21snmpd23:09:093
22482991515,0cyclictest771-21snmpd22:40:353
22482991515,0cyclictest771-21snmpd22:26:133
22482991515,0cyclictest771-21snmpd21:45:253
22482991515,0cyclictest771-21snmpd21:35:353
22482991515,0cyclictest771-21snmpd20:58:523
22482991515,0cyclictest771-21snmpd20:54:013
22482991515,0cyclictest771-21snmpd00:35:103
22474991515,0cyclictest66850irq/125-lan23:40:521
22474991515,0cyclictest66850irq/125-lan23:36:501
22474991515,0cyclictest66850irq/125-lan22:25:011
22474991515,0cyclictest66850irq/125-lan21:00:191
22474991514,1cyclictest66850irq/125-lan20:25:211
22474991514,1cyclictest66850irq/125-lan19:26:001
22482991414,0cyclictest771-21snmpd23:45:413
22482991414,0cyclictest771-21snmpd23:44:343
22482991414,0cyclictest771-21snmpd23:25:573
22482991414,0cyclictest771-21snmpd22:56:413
22482991414,0cyclictest771-21snmpd22:34:143
22482991414,0cyclictest771-21snmpd21:15:083
22482991414,0cyclictest771-21snmpd21:09:293
22482991414,0cyclictest771-21snmpd21:01:433
22482991414,0cyclictest771-21snmpd19:50:323
22482991414,0cyclictest771-21snmpd19:13:143
22474991414,0cyclictest66850irq/125-lan21:05:181
22474991414,0cyclictest66850irq/125-lan20:56:061
22474991414,0cyclictest66850irq/125-lan20:30:111
22474991414,0cyclictest66850irq/125-lan20:15:111
22474991414,0cyclictest66850irq/125-lan20:05:171
22474991413,1cyclictest66850irq/125-lan20:45:121
22482991313,0cyclictest771-21snmpd20:37:093
22482991313,0cyclictest771-21snmpd20:23:303
22482991313,0cyclictest771-21snmpd19:58:023
22482991313,0cyclictest771-21snmpd19:42:143
22482991313,0cyclictest771-21snmpd00:08:383
22482991313,0cyclictest771-21snmpd00:00:503
22474991313,0cyclictest66850irq/125-lan20:10:151
22474991312,1cyclictest66850irq/125-lan20:50:191
22474991312,1cyclictest66850irq/125-lan20:00:211
22474991212,0cyclictest8044-21cat20:20:161
22474991212,0cyclictest771-21snmpd19:47:561
22474991212,0cyclictest771-21snmpd19:22:231
22474991212,0cyclictest771-21snmpd19:11:351
2248299119,2cyclictest749-21ntpd22:18:323
22474991111,0cyclictest66850irq/125-lan19:19:221
224829998,1cyclictest769-21lldpd21:31:423
224829997,2cyclictest749-21ntpd21:43:503
224789998,1cyclictest771-21snmpd22:10:462
224789998,1cyclictest771-21snmpd22:03:462
224789998,1cyclictest771-21snmpd21:49:192
224789998,1cyclictest771-21snmpd20:13:062
224789998,1cyclictest517-21systemd-resolve22:48:362
224749999,0cyclictest66850irq/125-lan20:35:171
224829988,0cyclictest769-21lldpd22:22:143
224829986,2cyclictest749-21ntpd23:56:263
224829986,2cyclictest749-21ntpd23:37:303
224829986,2cyclictest749-21ntpd23:13:073
224829986,2cyclictest749-21ntpd22:50:273
224829986,2cyclictest749-21ntpd22:06:103
224829986,2cyclictest749-21ntpd22:04:553
224829986,2cyclictest749-21ntpd21:52:563
224829986,2cyclictest749-21ntpd21:13:583
224829986,2cyclictest749-21ntpd00:32:453
224829986,2cyclictest749-21ntpd00:12:183
224829980,8cyclictest28941-21cron23:20:013
224789988,0cyclictest769-21lldpd23:50:322
224789988,0cyclictest769-21lldpd22:56:372
224789988,0cyclictest769-21lldpd22:39:572
224789988,0cyclictest769-21lldpd22:24:062
224789988,0cyclictest769-21lldpd21:31:572
224789988,0cyclictest769-21lldpd21:26:042
224789988,0cyclictest769-21lldpd21:01:122
224789987,1cyclictest771-21snmpd23:44:402
224789987,1cyclictest771-21snmpd21:56:482
224789987,1cyclictest771-21snmpd20:30:272
224789987,1cyclictest771-21snmpd00:36:342
224789987,1cyclictest769-21lldpd21:44:342
224789986,2cyclictest771-21snmpd22:15:172
224789986,2cyclictest771-21snmpd20:36:482
224789986,2cyclictest771-21snmpd19:25:352
224749988,0cyclictest66850irq/125-lan19:40:501
224749988,0cyclictest66850irq/125-lan19:38:041
224699988,0cyclictest771-21snmpd23:45:040
224699988,0cyclictest771-21snmpd22:36:500
224699988,0cyclictest771-21snmpd21:48:510
224699988,0cyclictest771-21snmpd20:29:330
224699988,0cyclictest771-21snmpd20:21:450
224699988,0cyclictest771-21snmpd19:50:350
224699988,0cyclictest771-21snmpd19:10:290
224699988,0cyclictest771-21snmpd00:02:400
224699988,0cyclictest769-21lldpd22:16:440
224699988,0cyclictest769-21lldpd21:07:430
224699988,0cyclictest769-21lldpd20:59:430
224699988,0cyclictest769-21lldpd19:35:260
224699987,1cyclictest771-21snmpd20:30:510
224699987,1cyclictest771-21snmpd00:28:170
224699987,1cyclictest255-20systemd-journal21:44:530
224699986,2cyclictest771-21snmpd23:38:470
224699986,2cyclictest771-21snmpd21:52:030
224699985,2cyclictest771-21snmpd22:50:150
224699985,1cyclictest4088-21ssh22:22:160
224699984,3cyclictest255-20systemd-journal21:34:530
224829977,0cyclictest769-21lldpd21:58:243
224829977,0cyclictest769-21lldpd20:32:533
224829976,1cyclictest771-21snmpd20:48:133
224829976,1cyclictest749-21ntpd22:35:293
224829976,1cyclictest749-21ntpd00:19:373
224829976,1cyclictest24906-21cpuspeed_turbos00:20:113
224829975,2cyclictest771-21snmpd23:30:283
224829975,2cyclictest771-21snmpd20:12:483
224829975,2cyclictest771-21snmpd20:04:263
224829975,2cyclictest749-21ntpd23:00:063
224829975,2cyclictest749-21ntpd00:25:473
224829975,2cyclictest6002-21cat20:15:163
224829973,3cyclictest8589-21ssh21:22:383
224789977,0cyclictest771-21snmpd22:30:232
224789977,0cyclictest771-21snmpd19:53:172
224789977,0cyclictest771-21snmpd00:12:112
224789977,0cyclictest771-21snmpd00:05:502
224789977,0cyclictest769-21lldpd21:22:002
224789977,0cyclictest769-21lldpd21:15:582
224789977,0cyclictest769-21lldpd19:49:022
224789976,1cyclictest771-21snmpd23:20:302
224789975,2cyclictest771-21snmpd23:47:552
224789975,2cyclictest771-21snmpd22:09:342
224789975,2cyclictest771-21snmpd19:34:072
224789975,2cyclictest771-21snmpd00:20:152
224789973,3cyclictest255-20systemd-journal23:58:342
224699977,0cyclictest769-21lldpd23:02:110
224699977,0cyclictest769-21lldpd21:18:300
224699977,0cyclictest769-21lldpd19:33:540
224699977,0cyclictest769-21lldpd19:27:120
224699976,1cyclictest771-21snmpd23:05:240
224699976,1cyclictest771-21snmpd22:49:140
224699976,1cyclictest769-21lldpd20:16:250
224699976,1cyclictest524-21avahi-daemon22:25:510
224699976,1cyclictest524-21avahi-daemon00:10:000
224699975,2cyclictest771-21snmpd22:13:220
224699975,2cyclictest771-21snmpd00:10:380
224699973,3cyclictest255-20systemd-journal23:19:280
224699973,3cyclictest255-20systemd-journal21:23:390
224699972,5cyclictest550-21NetworkManager00:34:390
224829966,0cyclictest769-21lldpd20:42:033
224829965,1cyclictest749-21ntpd23:20:083
224829965,1cyclictest524-21avahi-daemon19:20:293
224829965,1cyclictest10293-21grep20:25:163
224829964,2cyclictest749-21ntpd21:29:043
224829962,3cyclictest28865-21ssh22:47:063
224789966,0cyclictest771-21snmpd19:18:592
224789966,0cyclictest769-21lldpd23:15:032
224789966,0cyclictest769-21lldpd23:09:272
224789966,0cyclictest769-21lldpd21:14:192
224789966,0cyclictest769-21lldpd00:31:022
224789966,0cyclictest769-21lldpd00:02:242
224789966,0cyclictest524-21avahi-daemon21:09:342
224789965,1cyclictest771-21snmpd20:17:362
224789965,1cyclictest769-21lldpd21:53:102
224789965,1cyclictest757-21cron20:45:002
224789965,1cyclictest32074-21ntp_kernel_pll_22:50:152
224789964,2cyclictest771-21snmpd22:26:522
224789964,2cyclictest771-21snmpd20:45:322
224789964,2cyclictest771-21snmpd20:28:002
224789963,3cyclictest550-21NetworkManager00:29:552
224699966,0cyclictest771-21snmpd22:00:090
224699966,0cyclictest771-21snmpd21:27:050
224699966,0cyclictest769-21lldpd20:43:590
224699966,0cyclictest769-21lldpd00:17:220
224699966,0cyclictest524-21avahi-daemon19:15:030
224699965,1cyclictest23877-21cpuspeed_turbos21:00:120
224699965,1cyclictest19514-21cpuspeed_turbos23:10:110
224699965,1cyclictest0-21swapper/000:20:170
224699963,3cyclictest550-21NetworkManager21:11:480
224699963,2cyclictest255-20systemd-journal23:56:130
224829955,0cyclictest524-21avahi-daemon19:48:313
224829953,2cyclictest749-21ntpd19:15:063
224829952,3cyclictest550-21NetworkManager19:26:063
224789954,1cyclictest21414-21cut00:15:182
224789953,2cyclictest771-21snmpd19:14:022
224789953,2cyclictest749-21ntpd20:20:052
224789953,2cyclictest550-21NetworkManager23:35:312
224789953,2cyclictest550-21NetworkManager20:00:232
224789953,2cyclictest20311-21ssh21:35:112
224789952,3cyclictest771-21snmpd19:36:402
224789952,3cyclictest517-21systemd-resolve20:57:022
224789952,2cyclictest8954-21ssh23:00:122
224789952,2cyclictest4177-21ssh23:25:552
224789952,2cyclictest25549-21ssh22:44:372
224789952,2cyclictest2256-21ntpq20:05:162
224789952,2cyclictest20905-21wc20:50:232
224789952,2cyclictest10854-21ssh23:33:252
224789950,4cyclictest20353-21ntpq23:10:172
224699955,0cyclictest769-21lldpd20:50:150
224699955,0cyclictest769-21lldpd19:46:200
224699955,0cyclictest769-21lldpd19:21:360
224699954,1cyclictest771-21snmpd19:40:520
224699953,2cyclictest8208-21ssh22:59:300
224699953,2cyclictest550-21NetworkManager19:55:130
224699953,2cyclictest255-20systemd-journal23:52:500
224699953,2cyclictest17859-21df_inode20:45:120
224699952,3cyclictest550-21NetworkManager20:07:090
224699952,2cyclictest9933-21ssh23:32:050
224699952,2cyclictest8319-21rm00:35:530
224699952,2cyclictest586-21rs:main0
224699952,2cyclictest32253-21gltestperf20:00:170
224699952,2cyclictest255-20systemd-journal20:12:400
224699952,2cyclictest22015-21ntp_192.168.11422:40:190
224699952,2cyclictest11764-21sort22:30:150
224699950,4cyclictest20480-21df_inode21:35:120
224829943,1cyclictest524-21avahi-daemon19:30:153
224829940,3cyclictest2304-21ntpq20:05:163
224789941,3cyclictest550-21NetworkManager19:40:072
224789941,2cyclictest586-21rs:main2
224789941,2cyclictest31093-21sed19:56:162
224699943,1cyclictest171ktimers/023:26:570
224699941,2cyclictest29813-21grep23:20:120
224699940,3cyclictest18143-21ntpq23:40:130
224699940,3cyclictest13658-21dpkg20:35:120
224829930,3cyclictest26532-21apt-config19:35:003
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional