You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-28 - 21:57

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack8slot6s.osadl.org (updated Fri Nov 28, 2025 12:43:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
16154991918,1cyclictest66850irq/125-lan11:45:173
16154991918,1cyclictest66850irq/125-lan10:35:143
16154991918,1cyclictest66850irq/125-lan10:00:373
16154991918,1cyclictest66850irq/125-lan09:50:123
16154991918,1cyclictest66850irq/125-lan09:25:153
16154991818,0cyclictest66850irq/125-lan11:23:473
16154991818,0cyclictest66850irq/125-lan09:57:433
16154991817,1cyclictest66850irq/125-lan12:34:203
16154991817,1cyclictest66850irq/125-lan12:20:123
16154991817,1cyclictest66850irq/125-lan12:10:183
16154991817,1cyclictest66850irq/125-lan11:43:053
16154991817,1cyclictest66850irq/125-lan11:15:123
16154991817,1cyclictest66850irq/125-lan11:09:553
16154991817,1cyclictest66850irq/125-lan10:40:113
16154991817,1cyclictest66850irq/125-lan10:05:173
16154991817,1cyclictest66850irq/125-lan09:40:163
16154991817,1cyclictest66850irq/125-lan09:30:423
16154991717,0cyclictest66850irq/125-lan12:15:063
16154991717,0cyclictest66850irq/125-lan11:30:143
16154991717,0cyclictest66850irq/125-lan11:10:393
16154991717,0cyclictest66850irq/125-lan11:00:183
16154991717,0cyclictest66850irq/125-lan10:59:493
16154991717,0cyclictest66850irq/125-lan10:50:123
16154991717,0cyclictest66850irq/125-lan10:45:363
16154991717,0cyclictest66850irq/125-lan10:23:443
16154991716,1cyclictest66850irq/125-lan12:35:283
16154991716,1cyclictest66850irq/125-lan12:25:293
16154991716,1cyclictest66850irq/125-lan12:07:163
16154991716,1cyclictest66850irq/125-lan12:04:303
16154991716,1cyclictest66850irq/125-lan11:58:423
16154991716,1cyclictest66850irq/125-lan11:39:163
16154991716,1cyclictest66850irq/125-lan11:25:153
16154991716,1cyclictest66850irq/125-lan10:25:203
16154991716,1cyclictest66850irq/125-lan10:11:403
16154991716,1cyclictest66850irq/125-lan09:45:273
16154991716,1cyclictest66850irq/125-lan09:10:383
16154991716,1cyclictest66850irq/125-lan07:50:163
16154991716,1cyclictest66850irq/125-lan07:10:133
16154991616,0cyclictest66850irq/125-lan07:45:183
16154991615,1cyclictest66850irq/125-lan11:51:243
16154991615,1cyclictest66850irq/125-lan10:31:093
16154991615,1cyclictest66850irq/125-lan10:15:123
16154991615,1cyclictest66850irq/125-lan09:20:013
16154991615,1cyclictest66850irq/125-lan09:16:403
16154991615,1cyclictest66850irq/125-lan08:35:093
16154991615,1cyclictest66850irq/125-lan07:20:163
16154991515,0cyclictest66850irq/125-lan09:35:203
16154991514,1cyclictest66850irq/125-lan08:30:143
16154991514,1cyclictest66850irq/125-lan08:28:163
16154991514,1cyclictest66850irq/125-lan08:20:163
16154991514,1cyclictest66850irq/125-lan07:35:203
16154991413,1cyclictest66850irq/125-lan08:45:143
16154991413,1cyclictest66850irq/125-lan08:40:183
16150991413,1cyclictest771-21snmpd11:02:322
16154991313,0cyclictest66850irq/125-lan08:55:143
16154991313,0cyclictest66850irq/125-lan08:00:123
16154991313,0cyclictest66850irq/125-lan07:55:113
16154991313,0cyclictest66850irq/125-lan07:43:263
16154991312,1cyclictest66850irq/125-lan08:50:203
16154991312,1cyclictest66850irq/125-lan08:10:193
16154991211,1cyclictest66850irq/125-lan09:05:003
16154991211,1cyclictest66850irq/125-lan07:15:013
16142991111,0cyclictest7082-21fwupd11:06:160
1615499109,1cyclictest66850irq/125-lan07:30:193
16154991010,0cyclictest66850irq/125-lan08:05:123
161549999,0cyclictest66850irq/125-lan08:15:133
161549999,0cyclictest66850irq/125-lan07:29:433
161509998,1cyclictest771-21snmpd09:14:092
161509998,1cyclictest771-21snmpd09:02:522
161509998,1cyclictest771-21snmpd08:17:572
161509998,1cyclictest769-21lldpd07:30:222
161459998,1cyclictest771-21snmpd12:38:071
161459998,1cyclictest771-21snmpd12:31:371
161429998,1cyclictest771-21snmpd09:45:410
161429998,1cyclictest771-21snmpd09:04:460
161429998,1cyclictest771-21snmpd08:31:060
161429998,1cyclictest769-21lldpd07:16:040
161429998,1cyclictest517-21systemd-resolve11:46:430
161429996,2cyclictest771-21snmpd10:30:490
161549987,1cyclictest66850irq/125-lan09:00:233
161509988,0cyclictest771-21snmpd12:25:152
161509988,0cyclictest771-21snmpd11:39:352
161509988,0cyclictest771-21snmpd11:29:192
161509988,0cyclictest771-21snmpd08:58:422
161509988,0cyclictest769-21lldpd12:12:572
161509988,0cyclictest769-21lldpd11:33:212
161509988,0cyclictest769-21lldpd10:55:542
161509988,0cyclictest769-21lldpd08:43:392
161509987,1cyclictest771-21snmpd09:55:502
161509987,1cyclictest771-21snmpd07:49:592
161509987,1cyclictest771-21snmpd07:42:282
161509987,1cyclictest517-21systemd-resolve10:27:282
161509986,2cyclictest17293-21cat10:05:142
161459988,0cyclictest771-21snmpd11:06:021
161459988,0cyclictest771-21snmpd10:33:041
161459988,0cyclictest771-21snmpd09:52:411
161459988,0cyclictest771-21snmpd08:10:351
161459988,0cyclictest769-21lldpd11:58:371
161459988,0cyclictest769-21lldpd10:50:131
161459988,0cyclictest769-21lldpd09:04:551
161459988,0cyclictest769-21lldpd08:24:381
161459987,1cyclictest771-21snmpd12:20:491
161459987,1cyclictest769-21lldpd11:42:551
161459987,1cyclictest769-21lldpd07:47:461
161459987,1cyclictest524-21avahi-daemon11:49:051
161459987,1cyclictest524-21avahi-daemon10:46:501
161459987,1cyclictest517-21systemd-resolve10:56:231
161459986,2cyclictest771-21snmpd12:01:301
161459986,2cyclictest771-21snmpd09:25:431
161459986,2cyclictest771-21snmpd07:57:101
161429988,0cyclictest771-21snmpd10:51:280
161429988,0cyclictest769-21lldpd12:23:200
161429988,0cyclictest769-21lldpd09:28:170
161429988,0cyclictest769-21lldpd07:47:100
161429987,1cyclictest771-21snmpd10:58:000
161429987,1cyclictest771-21snmpd08:22:080
161429987,1cyclictest524-21avahi-daemon10:25:540
161429986,2cyclictest771-21snmpd10:37:460
161429986,2cyclictest771-21snmpd09:59:200
161429986,2cyclictest771-21snmpd08:45:190
161509977,0cyclictest771-21snmpd12:23:222
161509977,0cyclictest771-21snmpd10:50:072
161509977,0cyclictest771-21snmpd09:39:042
161509977,0cyclictest771-21snmpd07:51:252
161509977,0cyclictest769-21lldpd10:30:362
161509977,0cyclictest769-21lldpd10:22:412
161509977,0cyclictest769-21lldpd10:17:362
161509977,0cyclictest769-21lldpd09:49:122
161509977,0cyclictest769-21lldpd09:15:272
161509977,0cyclictest769-21lldpd07:25:092
161509977,0cyclictest2587-21gltestperf11:40:142
161509976,1cyclictest771-21snmpd12:30:012
161509976,1cyclictest771-21snmpd12:02:482
161509976,1cyclictest771-21snmpd11:59:412
161509976,1cyclictest769-21lldpd08:36:382
161509976,1cyclictest524-21avahi-daemon11:52:332
161509976,1cyclictest255-20systemd-journal09:34:372
161509976,1cyclictest255-20systemd-journal08:14:572
161509976,1cyclictest255-20systemd-journal07:56:542
161509976,1cyclictest17471-21fschecks_time11:20:132
161509975,2cyclictest771-21snmpd11:47:032
161509974,3cyclictest771-21snmpd08:05:552
161459977,0cyclictest771-21snmpd10:39:011
161459977,0cyclictest771-21snmpd10:15:451
161459977,0cyclictest771-21snmpd08:47:191
161459977,0cyclictest771-21snmpd07:50:431
161459977,0cyclictest769-21lldpd11:16:311
161459977,0cyclictest769-21lldpd10:07:111
161459976,1cyclictest9669-21fschecks_count11:10:131
161459976,1cyclictest771-21snmpd11:39:111
161459976,1cyclictest771-21snmpd09:42:521
161459976,1cyclictest771-21snmpd07:24:271
161459976,1cyclictest769-21lldpd08:33:371
161459976,1cyclictest524-21avahi-daemon09:06:131
161459975,2cyclictest771-21snmpd09:47:381
161459975,2cyclictest771-21snmpd09:18:171
161459975,2cyclictest771-21snmpd07:18:131
161459975,2cyclictest771-21snmpd07:12:001
161459975,2cyclictest15131-21cat08:25:131
161429977,0cyclictest771-21snmpd12:10:560
161429977,0cyclictest771-21snmpd11:18:110
161429977,0cyclictest771-21snmpd10:45:460
161429977,0cyclictest771-21snmpd08:02:350
161429977,0cyclictest771-21snmpd07:34:160
161429977,0cyclictest771-21snmpd07:10:030
161429977,0cyclictest769-21lldpd12:35:060
161429977,0cyclictest769-21lldpd11:25:220
161429977,0cyclictest769-21lldpd11:20:530
161429977,0cyclictest769-21lldpd10:40:200
161429977,0cyclictest769-21lldpd09:08:350
161429977,0cyclictest769-21lldpd07:51:250
161429977,0cyclictest524-21avahi-daemon09:35:030
161429976,1cyclictest771-21snmpd10:04:270
161429976,1cyclictest771-21snmpd08:50:400
161429976,1cyclictest771-21snmpd08:44:030
161429976,1cyclictest771-21snmpd07:20:550
161429976,1cyclictest524-21avahi-daemon12:30:570
161429976,1cyclictest524-21avahi-daemon08:37:390
161429975,2cyclictest771-21snmpd10:19:240
161429975,2cyclictest771-21snmpd10:12:560
161429975,2cyclictest771-21snmpd07:43:040
161429975,2cyclictest17314-21cat10:05:150
161429973,3cyclictest7856-21ssh09:17:050
161429973,3cyclictest14856-21df08:25:110
161429973,3cyclictest14212-21awk12:30:000
161509966,0cyclictest769-21lldpd10:35:512
161509966,0cyclictest769-21lldpd08:02:162
161509966,0cyclictest524-21avahi-daemon07:36:022
161509965,1cyclictest771-21snmpd07:18:192
161509965,1cyclictest524-21avahi-daemon12:16:472
161509964,2cyclictest771-21snmpd08:52:152
161509964,2cyclictest771-21snmpd07:11:002
161509963,2cyclictest29324-21ssh09:41:552
161509962,4cyclictest550-21NetworkManager11:10:172
161509962,3cyclictest23007-21sh10:10:502
161509962,3cyclictest14136-21ssh10:01:372
161459966,0cyclictest771-21snmpd10:22:581
161459966,0cyclictest771-21snmpd09:33:411
161459966,0cyclictest769-21lldpd09:10:021
161459966,0cyclictest584-21in:imuxsock08:39:171
161459966,0cyclictest524-21avahi-daemon08:56:291
161459966,0cyclictest524-21avahi-daemon08:42:451
161459966,0cyclictest524-21avahi-daemon07:26:581
161459965,1cyclictest255-20systemd-journal10:03:531
161459964,2cyclictest771-21snmpd11:00:081
161459963,3cyclictest550-21NetworkManager07:30:311
161459963,3cyclictest517-21systemd-resolve12:09:591
161459963,2cyclictest255-20systemd-journal12:16:451
161459963,2cyclictest255-20systemd-journal11:52:111
161459962,3cyclictest30782-21ssh12:11:051
161459962,3cyclictest23026-21fw_conntrack09:35:141
161429966,0cyclictest771-21snmpd08:58:480
161429966,0cyclictest769-21lldpd09:24:410
161429966,0cyclictest769-21lldpd08:05:330
161429966,0cyclictest524-21avahi-daemon07:37:110
161429964,2cyclictest771-21snmpd07:28:360
161429964,2cyclictest586-21rs:main0
161429964,2cyclictest32615-21diskmemload11:35:240
161429964,1cyclictest771-21snmpd09:30:170
161429963,3cyclictest550-21NetworkManager11:10:550
161429962,3cyclictest4373-21grep07:59:590
161429962,3cyclictest31824-21ssh10:21:000
161509955,0cyclictest524-21avahi-daemon09:05:412
161509955,0cyclictest15050-21gltestperf08:25:122
161509953,2cyclictest771-21snmpd09:26:162
161509953,2cyclictest771-21snmpd08:48:492
161509952,3cyclictest550-21NetworkManager08:24:032
161509952,2cyclictest6127-21ssh09:53:012
161509952,2cyclictest26296-21ssh12:06:142
161509952,2cyclictest22209-21ssh10:46:422
161509952,2cyclictest20552-21seq12:36:012
161509952,2cyclictest20458-21apt-key07:19:592
161509952,2cyclictest16907-21sort08:30:122
161509952,2cyclictest14739-21ssh11:15:192
161509952,2cyclictest11027-21sed09:20:202
161509950,4cyclictest5636-21irqstats11:05:152
161459955,0cyclictest771-21snmpd10:28:451
161459953,2cyclictest771-21snmpd09:21:461
161459952,3cyclictest26348-21grep11:30:151
161459952,2cyclictest550-21NetworkManager08:05:201
161459952,2cyclictest27812-21wc07:35:181
161459952,2cyclictest26573-21basename08:54:591
161459952,2cyclictest255-20systemd-journal12:25:561
161459952,2cyclictest21721-21sed11:25:151
161459952,2cyclictest18869-21ssh11:21:001
161459952,2cyclictest17775-21ssh10:41:381
161429954,1cyclictest25837-21cpuspeed_turbos11:30:120
161429953,2cyclictest255-20systemd-journal11:52:330
161429952,3cyclictest550-21NetworkManager11:55:080
161429952,3cyclictest12522-21sed08:20:000
161429952,2cyclictest3482-21df09:50:130
161429952,2cyclictest3062-21grep11:40:170
161429952,2cyclictest2781-21ssh09:11:390
161429952,2cyclictest1692-21ld11:00:190
161429952,2cyclictest1118-21dpkg12:15:010
161429951,3cyclictest20636-21ntpq12:00:170
161429950,4cyclictest9406-21ntpq08:10:140
161429950,4cyclictest27906-21ntpq09:40:170
161509940,3cyclictest15834-21sh10:40:102
161459942,2cyclictest0-21swapper/107:40:121
161459940,4cyclictest5481-21ntpq08:00:181
161459940,4cyclictest11441-21ntpq08:15:171
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional