You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-06-28 - 20:41

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack8slot6s.osadl.org (updated Sat Jun 28, 2025 12:43:42)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
16431992120,1cyclictest66850irq/125-lan10:50:222
16431991918,1cyclictest66850irq/125-lan11:56:452
16431991918,1cyclictest66850irq/125-lan08:15:222
16431991818,0cyclictest66850irq/125-lan11:44:442
16431991818,0cyclictest66850irq/125-lan11:38:142
16431991818,0cyclictest66850irq/125-lan10:11:202
16431991818,0cyclictest66850irq/125-lan08:10:182
16431991817,1cyclictest66850irq/125-lan12:35:182
16431991817,1cyclictest66850irq/125-lan12:32:482
16431991817,1cyclictest66850irq/125-lan12:30:002
16431991817,1cyclictest66850irq/125-lan12:15:202
16431991817,1cyclictest66850irq/125-lan12:10:172
16431991817,1cyclictest66850irq/125-lan12:00:152
16431991817,1cyclictest66850irq/125-lan10:30:472
16431991817,1cyclictest66850irq/125-lan10:20:202
16431991817,1cyclictest66850irq/125-lan09:45:192
16431991817,1cyclictest66850irq/125-lan09:35:162
16431991817,1cyclictest66850irq/125-lan09:25:152
16431991817,1cyclictest66850irq/125-lan09:18:532
16431991817,1cyclictest66850irq/125-lan09:11:372
16431991817,1cyclictest66850irq/125-lan08:30:192
16431991817,1cyclictest66850irq/125-lan07:55:152
16431991717,0cyclictest66850irq/125-lan12:20:232
16431991717,0cyclictest66850irq/125-lan11:34:132
16431991717,0cyclictest66850irq/125-lan11:28:052
16431991717,0cyclictest66850irq/125-lan11:20:222
16431991717,0cyclictest66850irq/125-lan10:40:002
16431991717,0cyclictest66850irq/125-lan10:15:062
16431991717,0cyclictest66850irq/125-lan10:02:202
16431991717,0cyclictest66850irq/125-lan09:54:442
16431991717,0cyclictest66850irq/125-lan08:20:252
16431991716,1cyclictest66850irq/125-lan11:50:212
16431991716,1cyclictest66850irq/125-lan11:46:102
16431991716,1cyclictest66850irq/125-lan11:15:312
16431991716,1cyclictest66850irq/125-lan11:05:242
16431991716,1cyclictest66850irq/125-lan11:03:242
16431991716,1cyclictest66850irq/125-lan10:56:242
16431991716,1cyclictest66850irq/125-lan10:44:022
16431991716,1cyclictest66850irq/125-lan10:25:222
16431991716,1cyclictest66850irq/125-lan09:43:412
16431991716,1cyclictest66850irq/125-lan09:30:392
16431991716,1cyclictest66850irq/125-lan08:00:142
16431991716,1cyclictest66850irq/125-lan07:35:202
16431991716,1cyclictest66850irq/125-lan07:20:182
16431991716,1cyclictest66850irq/125-lan07:15:202
16431991616,0cyclictest66850irq/125-lan12:07:462
16431991616,0cyclictest66850irq/125-lan11:12:452
16431991616,0cyclictest66850irq/125-lan10:45:202
16431991615,1cyclictest66850irq/125-lan09:55:292
16431991615,1cyclictest66850irq/125-lan09:22:512
16431991615,1cyclictest66850irq/125-lan07:50:322
16431991515,0cyclictest66850irq/125-lan08:40:062
16431991515,0cyclictest66850irq/125-lan08:37:032
16431991515,0cyclictest66850irq/125-lan08:25:232
16431991514,1cyclictest66850irq/125-lan10:05:202
16431991514,1cyclictest66850irq/125-lan07:10:222
16431991414,0cyclictest66850irq/125-lan09:05:152
16431991413,1cyclictest66850irq/125-lan08:45:202
16431991313,0cyclictest66850irq/125-lan07:40:132
16431991313,0cyclictest66850irq/125-lan07:30:132
16431991211,1cyclictest66850irq/125-lan09:00:212
16431991211,1cyclictest66850irq/125-lan08:55:172
16431991211,1cyclictest66850irq/125-lan08:05:152
16431991211,1cyclictest66850irq/125-lan07:45:162
16434991111,0cyclictest771-21snmpd10:47:143
16434991111,0cyclictest771-21snmpd07:46:153
16434991111,0cyclictest771-21snmpd07:29:143
16434991010,0cyclictest771-21snmpd08:09:343
164349998,1cyclictest771-21snmpd11:31:353
164349998,1cyclictest769-21lldpd10:20:423
164279998,1cyclictest771-21snmpd08:06:461
164279998,1cyclictest524-21avahi-daemon10:50:491
164229998,1cyclictest771-21snmpd10:12:200
164229998,1cyclictest771-21snmpd07:41:000
164349988,0cyclictest771-21snmpd11:40:233
164349988,0cyclictest771-21snmpd11:18:043
164349988,0cyclictest771-21snmpd10:58:433
164349988,0cyclictest771-21snmpd09:36:133
164349988,0cyclictest769-21lldpd10:33:083
164349988,0cyclictest769-21lldpd08:12:443
164349988,0cyclictest769-21lldpd07:21:463
164349987,1cyclictest771-21snmpd11:07:013
164349987,1cyclictest771-21snmpd09:34:553
164349987,1cyclictest771-21snmpd08:17:403
164349987,1cyclictest771-21snmpd07:10:563
164349987,1cyclictest769-21lldpd11:46:593
164349986,2cyclictest771-21snmpd11:36:443
164349986,2cyclictest749-21ntpd12:35:023
164349986,2cyclictest749-21ntpd12:34:103
164349986,2cyclictest749-21ntpd12:15:113
164349986,2cyclictest32261-21cat11:00:203
164349984,4cyclictest550-21NetworkManager09:22:503
164319987,1cyclictest771-21snmpd08:50:262
164319987,1cyclictest66850irq/125-lan07:25:192
164279988,0cyclictest771-21snmpd09:53:011
164279988,0cyclictest771-21snmpd08:49:501
164279988,0cyclictest769-21lldpd11:52:291
164279988,0cyclictest769-21lldpd10:31:441
164279988,0cyclictest769-21lldpd09:18:521
164279988,0cyclictest769-21lldpd08:38:011
164279988,0cyclictest524-21avahi-daemon10:02:031
164279987,1cyclictest771-21snmpd09:42:161
164279987,1cyclictest771-21snmpd07:28:391
164279987,1cyclictest13068-21needreboot11:15:221
164279986,2cyclictest771-21snmpd12:10:511
164279986,2cyclictest771-21snmpd11:58:571
164279986,2cyclictest771-21snmpd11:39:071
164279986,2cyclictest771-21snmpd10:24:111
164279986,2cyclictest771-21snmpd09:04:051
164279986,2cyclictest771-21snmpd08:41:471
164229988,0cyclictest771-21snmpd11:20:330
164229988,0cyclictest771-21snmpd07:24:180
164229988,0cyclictest769-21lldpd12:33:480
164229988,0cyclictest769-21lldpd12:05:210
164229988,0cyclictest769-21lldpd12:03:110
164229988,0cyclictest769-21lldpd11:48:090
164229988,0cyclictest769-21lldpd09:11:430
164229988,0cyclictest517-21systemd-resolve11:16:260
164229987,1cyclictest771-21snmpd10:38:510
164229987,1cyclictest771-21snmpd08:18:210
164229987,1cyclictest771-21snmpd07:28:150
164229987,1cyclictest769-21lldpd09:21:210
164229987,1cyclictest769-21lldpd07:15:450
164229987,1cyclictest524-21avahi-daemon10:23:030
164229987,1cyclictest524-21avahi-daemon09:30:550
164229987,1cyclictest517-21systemd-resolve10:33:410
164229986,2cyclictest771-21snmpd11:25:320
164229986,2cyclictest771-21snmpd11:10:550
164229986,2cyclictest771-21snmpd10:18:460
164229986,2cyclictest771-21snmpd07:39:390
164349977,0cyclictest771-21snmpd10:25:383
164349977,0cyclictest769-21lldpd10:10:083
164349977,0cyclictest769-21lldpd09:57:393
164349977,0cyclictest769-21lldpd09:10:483
164349976,1cyclictest771-21snmpd10:52:183
164349976,1cyclictest771-21snmpd10:08:233
164349976,1cyclictest771-21snmpd09:53:313
164349976,1cyclictest771-21snmpd07:37:453
164349976,1cyclictest584-21in:imuxsock10:02:083
164349975,2cyclictest771-21snmpd12:06:123
164349975,2cyclictest771-21snmpd11:10:373
164349975,2cyclictest771-21snmpd07:52:163
164349975,2cyclictest749-21ntpd09:25:503
164279977,0cyclictest771-21snmpd12:38:251
164279977,0cyclictest771-21snmpd11:12:221
164279977,0cyclictest771-21snmpd10:10:441
164279977,0cyclictest771-21snmpd09:12:421
164279977,0cyclictest769-21lldpd09:58:391
164279976,1cyclictest771-21snmpd11:21:101
164279976,1cyclictest771-21snmpd10:35:061
164279976,1cyclictest771-21snmpd07:41:111
164279976,1cyclictest524-21avahi-daemon12:23:371
164279976,1cyclictest524-21avahi-daemon09:32:411
164279976,1cyclictest524-21avahi-daemon09:27:381
164279976,1cyclictest517-21systemd-resolve11:09:151
164279976,1cyclictest517-21systemd-resolve07:19:111
164279976,1cyclictest3274-21cat11:40:151
164279975,2cyclictest771-21snmpd10:45:571
164279975,2cyclictest771-21snmpd08:24:591
164279973,3cyclictest29536-21ssh10:56:311
164229977,0cyclictest771-21snmpd10:26:020
164229977,0cyclictest771-21snmpd08:58:030
164229977,0cyclictest771-21snmpd08:41:230
164229977,0cyclictest769-21lldpd07:10:460
164229976,1cyclictest771-21snmpd10:02:160
164229976,1cyclictest771-21snmpd09:52:190
164229976,1cyclictest769-21lldpd09:58:250
164229976,1cyclictest769-21lldpd08:23:350
164229976,1cyclictest524-21avahi-daemon12:26:490
164229976,1cyclictest524-21avahi-daemon09:15:530
164229976,1cyclictest517-21systemd-resolve08:14:340
164229975,2cyclictest771-21snmpd11:39:250
164229975,2cyclictest771-21snmpd11:09:220
164229975,2cyclictest771-21snmpd11:00:130
164229975,2cyclictest771-21snmpd08:27:560
164229975,2cyclictest771-21snmpd07:31:260
164229974,3cyclictest771-21snmpd10:05:500
164229974,2cyclictest255-20systemd-journal10:50:050
164349966,0cyclictest771-21snmpd10:16:503
164349966,0cyclictest771-21snmpd09:16:063
164349966,0cyclictest771-21snmpd08:56:163
164349966,0cyclictest771-21snmpd07:42:593
164349966,0cyclictest771-21snmpd07:18:363
164349966,0cyclictest769-21lldpd12:28:243
164349966,0cyclictest769-21lldpd08:42:173
164349966,0cyclictest524-21avahi-daemon08:25:033
164349965,1cyclictest771-21snmpd09:46:553
164349965,1cyclictest771-21snmpd09:06:483
164349965,1cyclictest749-21ntpd10:40:313
164349965,1cyclictest26313-21ntp_offset08:50:203
164349964,2cyclictest771-21snmpd12:02:063
164349964,2cyclictest771-21snmpd10:36:003
164349964,2cyclictest16754-21cat11:55:153
164349964,2cyclictest11337-21cat12:20:163
164349963,2cyclictest23175-21sed11:25:253
164349962,3cyclictest1871-21ssh12:10:483
164279966,0cyclictest771-21snmpd11:33:321
164279966,0cyclictest771-21snmpd07:33:361
164279966,0cyclictest524-21avahi-daemon07:49:541
164279966,0cyclictest524-21avahi-daemon07:13:141
164279965,1cyclictest771-21snmpd12:17:241
164279965,1cyclictest524-21avahi-daemon08:14:521
164279965,1cyclictest331ktimers/110:08:251
164279965,1cyclictest10069-21gltestperf09:20:141
164279964,2cyclictest771-21snmpd08:58:511
164279964,2cyclictest771-21snmpd07:20:361
164279963,2cyclictest255-20systemd-journal10:42:191
164279963,2cyclictest2083-21ssh11:02:421
164279962,3cyclictest24143-21users09:35:251
164229966,0cyclictest771-21snmpd07:58:040
164229966,0cyclictest771-21snmpd07:54:250
164229966,0cyclictest769-21lldpd11:55:510
164229966,0cyclictest524-21avahi-daemon09:42:560
164229965,1cyclictest771-21snmpd08:04:030
164229965,1cyclictest26357-21cpuspeed_turbos11:30:140
164229965,1cyclictest14953-21cut09:25:220
164229963,2cyclictest7179-21ssh12:15:430
164229962,3cyclictest678-21ssh09:45:330
164229960,6cyclictest0-21swapper/012:10:230
164349955,0cyclictest771-21snmpd11:20:013
164349955,0cyclictest771-21snmpd08:20:193
164349955,0cyclictest771-21snmpd08:00:103
164349955,0cyclictest771-21snmpd07:32:573
164349954,1cyclictest29762-21fschecks_time09:00:173
164349952,2cyclictest3682-21ntp_states07:55:183
164349951,3cyclictest28091-21ntpq09:40:223
164349951,3cyclictest23934-21irqstats08:45:173
164349951,3cyclictest19850-21ntpq08:35:233
164349951,3cyclictest14844-21ssh11:53:353
164279955,0cyclictest771-21snmpd07:35:001
164279955,0cyclictest769-21lldpd12:00:151
164279953,2cyclictest771-21snmpd08:25:051
164279953,2cyclictest255-20systemd-journal07:50:101
164279952,2cyclictest962-21ssh10:25:051
164279952,2cyclictest4363-21seq07:57:111
164279952,2cyclictest31730-21df09:45:141
164279952,2cyclictest27897-21ssh10:17:321
164279952,2cyclictest25488-21apt08:50:141
164279952,2cyclictest19303-21ssh12:27:301
164279951,3cyclictest23242-21ssh12:30:341
164279951,3cyclictest10233-21sh11:48:251
164279950,4cyclictest5590-21ntpq08:00:191
164279950,4cyclictest11709-21ntpq08:15:221
164279950,1cyclictest22194-21diskstats11:25:171
164229955,0cyclictest771-21snmpd10:45:090
164229955,0cyclictest771-21snmpd08:31:010
164229954,1cyclictest7777-21cut08:05:210
164229954,1cyclictest171ktimers/011:42:290
164229954,1cyclictest0-21swapper/010:55:160
164229953,2cyclictest771-21snmpd08:35:050
164229953,2cyclictest550-21NetworkManager12:35:230
164229953,2cyclictest11583-21grep12:20:180
164229952,3cyclictest511-21munin-plugin-st07:49:590
164229951,3cyclictest24026-21cat08:45:160
164229950,4cyclictest32367-21ntpq09:05:200
164229950,4cyclictest14986-21ntpq10:40:210
164349942,2cyclictest749-21ntpd08:30:123
164279941,3cyclictest32045-21cat09:05:181
164279941,2cyclictest27561-21cut12:05:181
164279940,3cyclictest17157-21diskstats08:30:151
164229943,1cyclictest171ktimers/009:00:150
164229943,1cyclictest0-21swapper/009:35:180
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional