You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-09-19 - 01:41

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack8slot6s.osadl.org (updated Thu Sep 18, 2025 12:43:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
5671991919,0cyclictest66850irq/125-lan07:55:000
5671991918,1cyclictest66850irq/125-lan12:14:030
5671991918,1cyclictest66850irq/125-lan11:58:420
5671991918,1cyclictest66850irq/125-lan11:49:410
5671991918,1cyclictest66850irq/125-lan10:55:000
5671991918,1cyclictest66850irq/125-lan10:44:060
5671991918,1cyclictest66850irq/125-lan10:30:120
5671991918,1cyclictest66850irq/125-lan10:00:040
5671991918,1cyclictest66850irq/125-lan09:57:460
5671991918,1cyclictest66850irq/125-lan09:29:060
5671991818,0cyclictest66850irq/125-lan11:36:340
5671991818,0cyclictest66850irq/125-lan11:32:210
5671991817,1cyclictest66850irq/125-lan12:37:040
5671991817,1cyclictest66850irq/125-lan12:07:240
5671991817,1cyclictest66850irq/125-lan12:00:190
5671991817,1cyclictest66850irq/125-lan11:50:130
5671991817,1cyclictest66850irq/125-lan11:44:240
5671991817,1cyclictest66850irq/125-lan11:26:370
5671991817,1cyclictest66850irq/125-lan11:24:200
5671991817,1cyclictest66850irq/125-lan11:13:220
5671991817,1cyclictest66850irq/125-lan10:46:130
5671991817,1cyclictest66850irq/125-lan10:27:030
5671991817,1cyclictest66850irq/125-lan10:17:540
5671991817,1cyclictest66850irq/125-lan10:09:450
5671991817,1cyclictest66850irq/125-lan09:43:260
5671991817,1cyclictest66850irq/125-lan09:34:090
5671991817,1cyclictest66850irq/125-lan09:10:170
5671991717,0cyclictest66850irq/125-lan12:31:090
5671991717,0cyclictest66850irq/125-lan11:06:060
5671991717,0cyclictest66850irq/125-lan10:55:220
5671991717,0cyclictest66850irq/125-lan08:55:160
5671991717,0cyclictest66850irq/125-lan08:15:210
5671991717,0cyclictest66850irq/125-lan07:50:150
5671991716,1cyclictest66850irq/125-lan12:25:180
5671991716,1cyclictest66850irq/125-lan12:19:000
5671991716,1cyclictest66850irq/125-lan11:15:210
5671991716,1cyclictest66850irq/125-lan10:35:130
5671991716,1cyclictest66850irq/125-lan09:50:180
5671991716,1cyclictest66850irq/125-lan09:45:160
5671991716,1cyclictest66850irq/125-lan09:20:130
5671991716,1cyclictest66850irq/125-lan09:18:570
5671991716,1cyclictest66850irq/125-lan08:40:200
5671991716,1cyclictest66850irq/125-lan08:25:100
5671991716,1cyclictest66850irq/125-lan07:35:210
5671991716,1cyclictest66850irq/125-lan07:30:190
5671991716,1cyclictest66850irq/125-lan07:25:210
5671991616,0cyclictest66850irq/125-lan10:10:460
5671991616,0cyclictest66850irq/125-lan08:45:140
5671991616,0cyclictest66850irq/125-lan07:14:350
5671991615,1cyclictest66850irq/125-lan12:22:530
5671991615,1cyclictest66850irq/125-lan11:02:180
5671991615,1cyclictest66850irq/125-lan09:37:440
5671991615,1cyclictest66850irq/125-lan08:00:190
5671991515,0cyclictest66850irq/125-lan10:22:540
5671991515,0cyclictest66850irq/125-lan09:00:100
5671991515,0cyclictest66850irq/125-lan08:05:140
5671991515,0cyclictest66850irq/125-lan07:45:120
5671991515,0cyclictest66850irq/125-lan07:15:210
5671991312,1cyclictest66850irq/125-lan08:35:180
5671991212,0cyclictest66850irq/125-lan07:20:120
5671991211,1cyclictest66850irq/125-lan08:20:190
5671991211,1cyclictest66850irq/125-lan07:40:090
5671991111,0cyclictest66850irq/125-lan08:30:220
5671991110,1cyclictest66850irq/125-lan09:10:000
568599108,2cyclictest769-21lldpd11:04:043
568599107,3cyclictest524-21avahi-daemon09:30:543
568599106,4cyclictest771-21snmpd11:37:583
568599106,4cyclictest771-21snmpd10:15:013
568599105,5cyclictest771-21snmpd12:04:113
568599102,5cyclictest550-21NetworkManager09:22:483
56859998,1cyclictest771-21snmpd12:18:413
56859998,1cyclictest771-21snmpd12:08:263
56859997,2cyclictest771-21snmpd07:54:493
56859997,2cyclictest769-21lldpd10:34:443
56859996,3cyclictest771-21snmpd10:56:053
56859996,3cyclictest771-21snmpd10:14:543
56859996,3cyclictest771-21snmpd07:14:183
56859996,3cyclictest769-21lldpd10:22:363
56859996,3cyclictest524-21avahi-daemon11:18:593
56859995,4cyclictest771-21snmpd11:07:003
56859993,3cyclictest550-21NetworkManager09:12:403
56859993,3cyclictest24692-21ssh09:45:533
56859993,2cyclictest32314-21ssh12:25:253
56859992,3cyclictest32445-21ssh11:25:453
56819998,1cyclictest771-21snmpd08:55:052
56819998,1cyclictest769-21lldpd12:01:332
56769998,1cyclictest769-21lldpd11:18:021
56769998,0cyclictest771-21snmpd11:37:251
56719998,1cyclictest66850irq/125-lan08:53:450
56719998,1cyclictest66850irq/125-lan08:10:170
56859988,0cyclictest771-21snmpd09:28:003
56859988,0cyclictest769-21lldpd12:35:403
56859988,0cyclictest769-21lldpd11:49:353
56859987,1cyclictest17759-21dump-pmu-power07:39:593
56859986,2cyclictest771-21snmpd09:58:453
56859986,2cyclictest771-21snmpd08:41:313
56859986,2cyclictest769-21lldpd08:03:123
56859986,2cyclictest749-21ntpd08:30:413
56859986,2cyclictest749-21ntpd08:26:193
56859986,2cyclictest749-21ntpd08:07:333
56859986,2cyclictest749-21ntpd07:25:043
56859985,3cyclictest771-21snmpd09:09:023
56859983,3cyclictest31624-21ntp_states11:55:193
56859983,2cyclictest550-21NetworkManager09:50:023
56859983,1cyclictest0-21swapper/312:20:213
56859982,3cyclictest10183-21perl07:20:123
56859982,2cyclictest9178-21sed08:35:183
56859982,2cyclictest586-21rs:main3
56859982,2cyclictest31119-21ssh09:19:413
56859982,2cyclictest21585-21ssh10:46:323
56859982,2cyclictest10834-21cat10:05:173
56859981,3cyclictest16875-21cat08:55:153
56819988,0cyclictest771-21snmpd11:40:252
56819988,0cyclictest771-21snmpd08:43:312
56819988,0cyclictest769-21lldpd11:02:162
56819988,0cyclictest769-21lldpd10:47:582
56819988,0cyclictest769-21lldpd10:32:582
56819988,0cyclictest769-21lldpd09:24:252
56819988,0cyclictest769-21lldpd08:20:452
56819988,0cyclictest769-21lldpd07:13:482
56819987,1cyclictest771-21snmpd11:56:432
56819987,1cyclictest771-21snmpd09:05:412
56819987,1cyclictest771-21snmpd07:33:562
56819987,1cyclictest771-21snmpd07:19:162
56819987,1cyclictest769-21lldpd10:09:092
56819987,1cyclictest769-21lldpd09:31:192
56819987,1cyclictest524-21avahi-daemon10:13:432
56819986,2cyclictest771-21snmpd12:17:412
56819986,2cyclictest771-21snmpd08:45:132
56819985,3cyclictest771-21snmpd09:53:372
56769988,0cyclictest771-21snmpd12:17:051
56769988,0cyclictest771-21snmpd09:05:261
56769988,0cyclictest769-21lldpd10:43:281
56769988,0cyclictest769-21lldpd10:05:431
56769987,1cyclictest771-21snmpd12:07:381
56769987,1cyclictest771-21snmpd08:26:361
56769987,1cyclictest771-21snmpd08:21:271
56769987,1cyclictest769-21lldpd11:20:581
56769987,1cyclictest769-21lldpd09:47:231
56769986,2cyclictest771-21snmpd10:57:301
56769986,2cyclictest771-21snmpd09:58:491
56769986,2cyclictest771-21snmpd09:35:541
56769986,2cyclictest771-21snmpd08:38:551
56769986,2cyclictest771-21snmpd07:36:171
56769986,2cyclictest749-21ntpd10:21:491
56769986,2cyclictest749-21ntpd09:33:341
56769986,2cyclictest4393-21cat09:25:141
56859977,0cyclictest771-21snmpd10:25:193
56859977,0cyclictest769-21lldpd11:12:233
56859977,0cyclictest524-21avahi-daemon07:44:043
56859975,2cyclictest771-21snmpd08:23:473
56859973,3cyclictest550-21NetworkManager12:10:283
56859973,3cyclictest26306-21ssh10:50:393
56859972,3cyclictest550-21NetworkManager11:40:253
56859972,2cyclictest8999-21ls07:15:213
56859972,2cyclictest771-21snmpd10:40:283
56859972,2cyclictest771-21snmpd08:10:063
56859972,1cyclictest0-21swapper/309:00:113
56859971,3cyclictest15390-21sed08:50:203
56859971,2cyclictest5846-21ssh12:31:063
56859971,2cyclictest28619-21ssh11:52:473
56819977,0cyclictest771-21snmpd11:26:522
56819977,0cyclictest771-21snmpd09:42:112
56819977,0cyclictest771-21snmpd09:39:542
56819977,0cyclictest771-21snmpd07:28:212
56819977,0cyclictest769-21lldpd12:26:532
56819977,0cyclictest769-21lldpd11:35:192
56819977,0cyclictest769-21lldpd10:59:022
56819977,0cyclictest769-21lldpd10:23:222
56819977,0cyclictest769-21lldpd09:49:352
56819976,1cyclictest771-21snmpd11:54:262
56819976,1cyclictest769-21lldpd09:18:032
56819976,1cyclictest769-21lldpd07:54:302
56819976,1cyclictest524-21avahi-daemon11:23:132
56819976,1cyclictest524-21avahi-daemon07:39:452
56819976,1cyclictest517-21systemd-resolve12:30:212
56819976,1cyclictest255-20systemd-journal10:40:342
56819975,2cyclictest771-21snmpd12:11:142
56819975,2cyclictest771-21snmpd07:21:582
56819974,3cyclictest550-21NetworkManager09:25:432
56819974,2cyclictest771-21snmpd08:32:102
56819973,3cyclictest15182-21ntp_states08:50:182
56769977,0cyclictest771-21snmpd11:29:191
56769977,0cyclictest771-21snmpd10:01:491
56769977,0cyclictest771-21snmpd08:44:241
56769977,0cyclictest771-21snmpd08:10:231
56769977,0cyclictest771-21snmpd08:03:481
56769977,0cyclictest771-21snmpd07:45:381
56769977,0cyclictest769-21lldpd09:40:221
56769977,0cyclictest26744-21unixbench_singl11:50:221
56769976,1cyclictest771-21snmpd12:21:241
56769976,1cyclictest771-21snmpd11:33:431
56769976,1cyclictest771-21snmpd11:08:481
56769975,2cyclictest771-21snmpd08:31:101
56769975,2cyclictest771-21snmpd08:16:091
56769975,2cyclictest749-21ntpd11:59:271
56769975,2cyclictest749-21ntpd10:33:111
56769975,2cyclictest749-21ntpd10:11:381
56769973,3cyclictest27050-21apt-get09:15:161
56859966,0cyclictest769-21lldpd09:35:573
56859964,2cyclictest749-21ntpd07:55:103
56859963,1cyclictest18447-21gltestperf09:40:163
56859962,2cyclictest8167-21ssh11:34:183
56819966,0cyclictest769-21lldpd12:22:112
56819966,0cyclictest769-21lldpd07:59:402
56819966,0cyclictest517-21systemd-resolve11:09:372
56819965,1cyclictest19038-21cut09:00:142
56819964,2cyclictest771-21snmpd11:10:392
56819963,2cyclictest255-20systemd-journal11:46:542
56819962,3cyclictest24659-21ssh09:11:272
56769966,0cyclictest771-21snmpd11:05:001
56769966,0cyclictest771-21snmpd07:22:281
56769966,0cyclictest524-21avahi-daemon09:22:221
56769966,0cyclictest517-21systemd-resolve10:37:231
56769966,0cyclictest28801-21needreboot08:05:171
56769965,1cyclictest749-21ntpd12:30:561
56769965,1cyclictest749-21ntpd10:48:031
56769964,2cyclictest771-21snmpd09:01:291
56769964,2cyclictest550-21NetworkManager10:26:591
56769963,3cyclictest550-21NetworkManager12:02:481
56769963,2cyclictest31279-21ssh12:25:161
56769963,2cyclictest18504-21ssh11:14:321
56859955,0cyclictest9488-21ssh10:35:183
56859952,2cyclictest6220-21ssh10:00:203
56859952,2cyclictest25665-21ls11:20:153
56859952,2cyclictest15109-21cat07:30:183
56859951,4cyclictest550-21NetworkManager08:46:553
56819955,0cyclictest771-21snmpd08:14:212
56819955,0cyclictest769-21lldpd07:45:392
56819954,1cyclictest25419-21ntpq10:50:152
56819953,2cyclictest771-21snmpd12:05:472
56819953,2cyclictest771-21snmpd08:01:152
56819953,2cyclictest550-21NetworkManager08:25:172
56819952,2cyclictest8934-21cut10:35:142
56819952,2cyclictest6737-21ssh10:00:372
56819952,2cyclictest586-21rs:main2
56819952,2cyclictest255-20systemd-journal10:25:382
56819952,2cyclictest255-20systemd-journal08:06:052
56819952,2cyclictest23136-21ssh11:17:522
56819951,3cyclictest510-21grep09:55:132
56819951,3cyclictest255-20systemd-journal12:35:472
56769955,0cyclictest524-21avahi-daemon08:46:511
56769954,1cyclictest331ktimers/109:50:121
56769954,1cyclictest24159-21cpuspeed_turbos07:55:151
56769952,2cyclictest27672-21kworker/u16:2+nfsiod11:45:141
56769952,2cyclictest26629-21ssh10:51:031
56769952,2cyclictest23234-21kworker/u16:1+nfsiod09:10:171
56769952,2cyclictest21844-21rm10:15:511
56769952,2cyclictest14467-21ssh11:40:071
56769952,2cyclictest10704-21sed12:35:271
56769951,3cyclictest8277-21ls07:15:131
56769950,3cyclictest16285-21cat08:55:001
56859942,2cyclictest891-21sed08:15:203
56819941,2cyclictest8438-21df_inode08:35:132
56819941,2cyclictest3670-21ssh11:30:102
56769942,2cyclictest771-21snmpd07:10:061
56769942,2cyclictest16563-21cut08:55:121
56769942,2cyclictest13584-21seq07:29:021
56769941,2cyclictest586-21rs:main1
56769941,2cyclictest255-20systemd-journal07:43:031
56769941,0cyclictest141642cut07:30:111
56769940,3cyclictest15340-21ntpq12:10:161
56819930,2cyclictest18130-21cpuspeed_turbos07:40:112
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional