You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-21 - 18:35

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack8slot6s.osadl.org (updated Fri Nov 21, 2025 12:43:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
16070992019,1cyclictest66850irq/125-lan10:52:063
16070992018,2cyclictest66850irq/125-lan10:00:113
16070991918,1cyclictest66850irq/125-lan11:37:313
16070991918,1cyclictest66850irq/125-lan10:44:033
16070991918,1cyclictest66850irq/125-lan09:45:303
16070991917,2cyclictest66850irq/125-lan12:30:113
16070991917,2cyclictest66850irq/125-lan09:05:193
16070991917,2cyclictest66850irq/125-lan07:15:143
16070991916,3cyclictest66850irq/125-lan11:56:463
16070991818,0cyclictest9485-21fwupd11:06:193
16070991818,0cyclictest66850irq/125-lan12:01:513
16070991818,0cyclictest66850irq/125-lan11:33:553
16070991818,0cyclictest66850irq/125-lan11:20:143
16070991818,0cyclictest66850irq/125-lan10:28:503
16070991818,0cyclictest66850irq/125-lan09:24:263
16070991817,1cyclictest66850irq/125-lan12:14:073
16070991817,1cyclictest66850irq/125-lan12:07:253
16070991817,1cyclictest66850irq/125-lan11:40:213
16070991817,1cyclictest66850irq/125-lan10:45:183
16070991817,1cyclictest66850irq/125-lan08:10:073
16070991816,2cyclictest66850irq/125-lan12:35:183
16070991816,2cyclictest66850irq/125-lan11:48:263
16070991816,2cyclictest66850irq/125-lan11:12:073
16070991816,2cyclictest66850irq/125-lan10:56:073
16070991816,2cyclictest66850irq/125-lan10:36:163
16070991816,2cyclictest66850irq/125-lan10:30:213
16070991816,2cyclictest66850irq/125-lan09:35:153
16070991816,2cyclictest66850irq/125-lan09:19:443
16062991818,0cyclictest771-21snmpd11:45:361
16062991818,0cyclictest771-21snmpd09:48:051
16062991818,0cyclictest771-21snmpd09:42:481
16062991818,0cyclictest771-21snmpd08:41:341
16062991818,0cyclictest20724-21cat11:55:121
16062991818,0cyclictest18318-21cat08:30:151
16070991717,0cyclictest66850irq/125-lan11:03:443
16070991717,0cyclictest66850irq/125-lan10:20:123
16070991717,0cyclictest66850irq/125-lan08:00:163
16070991716,1cyclictest66850irq/125-lan12:25:063
16070991716,1cyclictest66850irq/125-lan12:17:283
16070991716,1cyclictest66850irq/125-lan10:16:363
16070991716,1cyclictest66850irq/125-lan10:10:163
16070991716,1cyclictest66850irq/125-lan09:50:133
16070991716,1cyclictest66850irq/125-lan09:30:153
16070991716,1cyclictest66850irq/125-lan09:10:153
16070991715,2cyclictest66850irq/125-lan12:22:253
16070991715,2cyclictest66850irq/125-lan11:16:003
16070991715,2cyclictest66850irq/125-lan07:10:203
16062991717,0cyclictest771-21snmpd12:30:101
16062991717,0cyclictest771-21snmpd12:00:571
16062991717,0cyclictest771-21snmpd11:52:261
16062991717,0cyclictest771-21snmpd11:34:341
16062991717,0cyclictest771-21snmpd10:54:181
16062991717,0cyclictest771-21snmpd10:01:021
16062991717,0cyclictest771-21snmpd09:51:281
16062991717,0cyclictest771-21snmpd07:27:411
16070991616,0cyclictest66850irq/125-lan11:51:343
16070991616,0cyclictest66850irq/125-lan08:43:213
16070991615,1cyclictest66850irq/125-lan09:25:143
16070991615,1cyclictest66850irq/125-lan08:45:153
16070991615,1cyclictest66850irq/125-lan08:39:003
16070991614,2cyclictest66850irq/125-lan10:05:033
16070991614,2cyclictest66850irq/125-lan09:40:043
16070991515,0cyclictest66850irq/125-lan08:55:153
16070991515,0cyclictest66850irq/125-lan07:40:003
16070991514,1cyclictest66850irq/125-lan11:25:133
16070991514,1cyclictest66850irq/125-lan08:20:153
16070991513,2cyclictest66850irq/125-lan07:20:183
16070991413,1cyclictest66850irq/125-lan09:56:403
16070991413,1cyclictest66850irq/125-lan07:35:213
16070991412,2cyclictest66850irq/125-lan07:56:203
16070991313,0cyclictest66850irq/125-lan08:25:173
16070991212,0cyclictest66850irq/125-lan08:05:083
16070991211,1cyclictest66850irq/125-lan08:30:183
16062991211,1cyclictest771-21snmpd10:35:231
16070991111,0cyclictest66850irq/125-lan08:50:143
16070991111,0cyclictest66850irq/125-lan07:50:233
16070991110,1cyclictest66850irq/125-lan08:16:473
16070991110,1cyclictest66850irq/125-lan07:25:133
1607099109,1cyclictest66850irq/125-lan07:45:153
160709999,0cyclictest66850irq/125-lan09:00:153
160709998,1cyclictest66850irq/125-lan07:30:153
160659998,1cyclictest771-21snmpd11:18:382
160659998,1cyclictest771-21snmpd10:20:352
160659998,1cyclictest771-21snmpd09:07:202
160659998,1cyclictest24301-21ntp_kernel_err10:10:152
160659997,2cyclictest771-21snmpd09:56:432
160659997,1cyclictest771-21snmpd11:21:162
160629998,1cyclictest771-21snmpd08:13:251
160629996,3cyclictest771-21snmpd08:55:021
160629996,3cyclictest749-21ntpd12:06:021
160579998,1cyclictest771-21snmpd09:50:400
160579998,1cyclictest771-21snmpd09:37:570
160579998,1cyclictest771-21snmpd08:24:040
160579998,1cyclictest771-21snmpd07:52:390
160579998,1cyclictest769-21lldpd11:24:270
160659988,0cyclictest771-21snmpd11:25:342
160659988,0cyclictest769-21lldpd11:55:502
160659988,0cyclictest769-21lldpd09:39:222
160659988,0cyclictest769-21lldpd08:41:592
160659988,0cyclictest769-21lldpd07:14:262
160659987,1cyclictest771-21snmpd11:10:152
160659987,1cyclictest771-21snmpd08:47:172
160659987,1cyclictest769-21lldpd08:24:452
160659987,1cyclictest524-21avahi-daemon09:25:172
160659986,2cyclictest771-21snmpd08:18:222
160629988,0cyclictest771-21snmpd10:55:181
160629988,0cyclictest771-21snmpd08:51:441
160629988,0cyclictest769-21lldpd11:06:411
160629988,0cyclictest769-21lldpd07:31:181
160629987,1cyclictest771-21snmpd11:17:111
160629987,1cyclictest524-21avahi-daemon11:44:591
160629987,1cyclictest524-21avahi-daemon10:20:411
160629987,1cyclictest13572-21idleruntime-cro08:20:011
160629986,2cyclictest749-21ntpd12:39:291
160629986,2cyclictest749-21ntpd11:29:181
160629986,2cyclictest749-21ntpd11:02:301
160629986,2cyclictest749-21ntpd10:45:031
160629986,2cyclictest749-21ntpd10:43:471
160629986,2cyclictest749-21ntpd10:31:011
160629986,2cyclictest749-21ntpd10:05:021
160629986,2cyclictest749-21ntpd09:35:201
160629986,2cyclictest749-21ntpd09:31:371
160629986,2cyclictest749-21ntpd09:20:361
160629986,2cyclictest749-21ntpd09:15:311
160629986,2cyclictest749-21ntpd09:10:401
160629984,3cyclictest255-20systemd-journal10:19:261
160579988,0cyclictest771-21snmpd12:39:400
160579988,0cyclictest771-21snmpd11:08:180
160579988,0cyclictest771-21snmpd09:31:570
160579988,0cyclictest769-21lldpd12:22:510
160579988,0cyclictest769-21lldpd11:53:110
160579988,0cyclictest769-21lldpd11:31:090
160579988,0cyclictest769-21lldpd10:41:210
160579988,0cyclictest769-21lldpd10:30:590
160579987,1cyclictest771-21snmpd08:59:410
160579987,1cyclictest524-21avahi-daemon11:56:050
160579987,1cyclictest517-21systemd-resolve12:28:030
160579986,2cyclictest771-21snmpd08:50:560
160579986,2cyclictest771-21snmpd08:18:410
160579986,2cyclictest771-21snmpd08:06:120
160579986,2cyclictest771-21snmpd08:03:120
160579986,2cyclictest771-21snmpd07:20:290
160579986,2cyclictest771-21snmpd07:11:320
160579986,2cyclictest2469-21cat11:35:150
160659977,0cyclictest771-21snmpd09:48:022
160659977,0cyclictest769-21lldpd11:50:562
160659977,0cyclictest769-21lldpd10:50:032
160659977,0cyclictest769-21lldpd09:12:512
160659976,1cyclictest771-21snmpd10:28:382
160659976,1cyclictest771-21snmpd09:21:542
160659976,1cyclictest771-21snmpd07:58:182
160659976,1cyclictest584-21in:imuxsock12:20:452
160659976,1cyclictest517-21systemd-resolve10:17:542
160659976,1cyclictest517-21systemd-resolve10:00:402
160659975,2cyclictest771-21snmpd10:49:282
160659975,2cyclictest771-21snmpd09:32:522
160659975,2cyclictest771-21snmpd09:16:192
160659975,2cyclictest771-21snmpd08:39:492
160659975,2cyclictest771-21snmpd08:10:242
160659975,2cyclictest1349-21sh12:40:002
160659973,3cyclictest255-20systemd-journal12:15:042
160629977,0cyclictest771-21snmpd12:21:191
160629977,0cyclictest771-21snmpd07:23:381
160629977,0cyclictest769-21lldpd10:10:061
160629977,0cyclictest769-21lldpd08:21:491
160629977,0cyclictest769-21lldpd07:46:011
160629976,1cyclictest749-21ntpd10:26:391
160629976,1cyclictest612-21gmain12:28:331
160629976,1cyclictest524-21avahi-daemon08:35:121
160629975,2cyclictest771-21snmpd09:06:201
160629975,2cyclictest771-21snmpd07:18:481
160629975,2cyclictest749-21ntpd12:16:561
160629975,2cyclictest749-21ntpd09:55:201
160629974,3cyclictest550-21NetworkManager09:27:351
160579977,0cyclictest771-21snmpd11:14:160
160579977,0cyclictest769-21lldpd10:26:180
160579977,0cyclictest769-21lldpd09:57:380
160579977,0cyclictest769-21lldpd09:07:230
160579977,0cyclictest769-21lldpd09:01:510
160579976,1cyclictest769-21lldpd10:08:440
160579976,1cyclictest769-21lldpd09:17:350
160579976,1cyclictest30703-21fschecks_count12:05:160
160579975,2cyclictest771-21snmpd11:40:130
160579975,2cyclictest771-21snmpd11:25:430
160579975,2cyclictest771-21snmpd10:00:580
160579975,2cyclictest32121-21cat07:45:120
160579972,4cyclictest31182-21cat10:55:000
160659965,1cyclictest771-21snmpd09:01:502
160659965,1cyclictest524-21avahi-daemon12:00:562
160659965,1cyclictest517-21systemd-resolve10:36:142
160659963,3cyclictest7036-21head11:40:142
160659963,3cyclictest5651-21fw_forwarded_lo08:00:142
160659963,3cyclictest550-21NetworkManager07:50:192
160659963,3cyclictest18054-21diskstats08:30:132
160659963,2cyclictest8275-21ntpq11:05:162
160659963,2cyclictest586-21rs:main2
160659963,2cyclictest25246-21ssh12:31:172
160659963,2cyclictest23705-21ps07:25:162
160659962,3cyclictest4628-21ssh11:37:172
160659962,3cyclictest31881-21sed12:05:232
160659962,3cyclictest30916-21sed07:40:242
160659962,3cyclictest19694-21ntpq10:05:182
160629966,0cyclictest771-21snmpd11:22:461
160629965,1cyclictest771-21snmpd07:12:471
160629965,1cyclictest769-21lldpd08:25:311
160629965,1cyclictest769-21lldpd08:02:511
160629965,1cyclictest749-21ntpd07:35:001
160629964,2cyclictest771-21snmpd07:40:511
160629963,3cyclictest771-21snmpd09:04:541
160629963,2cyclictest3091-21grep12:10:121
160629962,3cyclictest13841-21ssh11:10:521
160579966,0cyclictest771-21snmpd12:18:090
160579966,0cyclictest769-21lldpd10:18:580
160579966,0cyclictest769-21lldpd08:41:410
160579966,0cyclictest524-21avahi-daemon07:44:090
160579965,1cyclictest22989-21cat10:45:140
160579964,2cyclictest771-21snmpd08:10:270
160579963,3cyclictest550-21NetworkManager11:46:170
160579963,3cyclictest550-21NetworkManager11:19:180
160579963,3cyclictest550-21NetworkManager09:23:470
160579963,3cyclictest4104-21ntp_kernel_pll_07:55:160
160579962,3cyclictest26012-21memory12:00:160
160579962,3cyclictest24237-21fw_conntrack08:45:140
160659954,1cyclictest524-21avahi-daemon07:33:532
160659954,1cyclictest421ktimers/210:43:532
160659954,1cyclictest10902-21tr11:45:142
160659954,1cyclictest0-21swapper/211:30:212
160659953,2cyclictest771-21snmpd10:32:302
160659953,2cyclictest771-21snmpd07:48:272
160659953,2cyclictest586-21rs:main2
160659953,2cyclictest4140-21sshd11:00:212
160659952,3cyclictest28744-21ld08:55:172
160659952,2cyclictest7639-21ssh09:52:182
160659952,2cyclictest28777-21kworker/u16:2+nfsiod12:13:082
160659952,2cyclictest20463-21forks07:20:112
160659952,2cyclictest1370-21ssh10:57:072
160659950,4cyclictest27881-21ntpq07:35:172
160629954,1cyclictest771-21snmpd07:57:001
160629952,3cyclictest550-21NetworkManager08:05:011
160629952,2cyclictest1603-21fw_conntrack07:50:141
160629951,3cyclictest255-20systemd-journal08:45:291
160579954,1cyclictest23876-21ntpq12:30:180
160579954,1cyclictest171ktimers/010:14:480
160579953,2cyclictest550-21NetworkManager09:41:250
160579953,2cyclictest17888-21apt-get08:30:140
160579952,3cyclictest550-21NetworkManager08:26:420
160579952,3cyclictest2938-21apt-get12:10:120
160579952,2cyclictest5777-21ssh11:03:110
160579952,2cyclictest451-21ssh10:55:340
160579952,2cyclictest3768-21ssh09:10:370
160579952,2cyclictest14292-21missed_timers10:35:160
160579952,2cyclictest1265-21ssh09:45:170
160659942,2cyclictest771-21snmpd08:25:042
160659942,2cyclictest7683-21gltestperf08:05:112
160659942,2cyclictest26387-21cat08:50:142
160659940,3cyclictest18949-21ntp_192.168.11407:15:162
160579943,1cyclictest15231-21cpuspeed_turbos09:25:110
160579942,2cyclictest749-21ntpd07:30:100
160579941,2cyclictest32626-21sh10:20:130
160579941,2cyclictest27839-21strings07:35:160
160579941,2cyclictest22355-21apt-config07:25:000
160579940,3cyclictest19904-21runrttasks08:35:120
160579940,3cyclictest18418-21diskstats07:15:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional