You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-02 - 18:36

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack8slot6s.osadl.org (updated Tue Dec 02, 2025 12:43:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
20227991918,1cyclictest66850irq/125-lan09:29:121
20227991818,0cyclictest771-21snmpd12:36:081
20227991818,0cyclictest771-21snmpd11:15:531
20227991818,0cyclictest771-21snmpd10:05:081
20227991818,0cyclictest771-21snmpd09:09:401
20227991818,0cyclictest771-21snmpd08:08:381
20227991818,0cyclictest771-21snmpd08:01:491
20227991818,0cyclictest66850irq/125-lan12:09:481
20227991818,0cyclictest66850irq/125-lan12:03:161
20227991818,0cyclictest66850irq/125-lan11:35:151
20227991818,0cyclictest66850irq/125-lan10:51:231
20227991818,0cyclictest66850irq/125-lan10:45:221
20227991818,0cyclictest66850irq/125-lan10:25:181
20227991818,0cyclictest66850irq/125-lan10:22:431
20227991818,0cyclictest66850irq/125-lan10:12:181
20227991817,1cyclictest66850irq/125-lan12:20:191
20227991817,1cyclictest66850irq/125-lan11:51:381
20227991817,1cyclictest66850irq/125-lan11:31:401
20227991817,1cyclictest66850irq/125-lan11:21:241
20227991817,1cyclictest66850irq/125-lan11:08:361
20227991817,1cyclictest66850irq/125-lan10:32:351
20227991817,1cyclictest66850irq/125-lan10:15:191
20227991817,1cyclictest66850irq/125-lan10:00:141
20227991817,1cyclictest66850irq/125-lan09:38:021
20227991817,1cyclictest66850irq/125-lan09:20:551
20227991717,0cyclictest771-21snmpd11:04:561
20227991717,0cyclictest771-21snmpd10:39:431
20227991717,0cyclictest771-21snmpd08:53:511
20227991717,0cyclictest771-21snmpd08:38:121
20227991717,0cyclictest771-21snmpd08:27:081
20227991717,0cyclictest66850irq/125-lan12:30:191
20227991717,0cyclictest66850irq/125-lan12:19:481
20227991717,0cyclictest66850irq/125-lan12:11:541
20227991717,0cyclictest66850irq/125-lan11:56:291
20227991717,0cyclictest66850irq/125-lan11:43:051
20227991717,0cyclictest66850irq/125-lan11:10:141
20227991717,0cyclictest66850irq/125-lan09:58:041
20227991717,0cyclictest66850irq/125-lan09:47:281
20227991717,0cyclictest66850irq/125-lan09:10:301
20227991717,0cyclictest66850irq/125-lan08:15:151
20227991717,0cyclictest66850irq/125-lan08:10:151
20227991717,0cyclictest66850irq/125-lan07:30:191
20227991717,0cyclictest66850irq/125-lan07:25:131
20227991716,1cyclictest66850irq/125-lan12:25:391
20227991716,1cyclictest66850irq/125-lan11:46:541
20227991716,1cyclictest66850irq/125-lan11:25:151
20227991716,1cyclictest66850irq/125-lan10:59:061
20227991716,1cyclictest66850irq/125-lan10:41:481
20227991716,1cyclictest66850irq/125-lan09:50:211
20227991716,1cyclictest66850irq/125-lan08:55:141
20227991716,1cyclictest66850irq/125-lan08:40:231
20227991716,1cyclictest66850irq/125-lan07:57:421
20227991616,0cyclictest66850irq/125-lan09:30:151
20227991616,0cyclictest66850irq/125-lan09:15:111
20227991616,0cyclictest66850irq/125-lan08:32:531
20227991616,0cyclictest66850irq/125-lan07:40:151
20227991515,0cyclictest66850irq/125-lan09:42:231
20227991515,0cyclictest66850irq/125-lan08:20:131
20227991515,0cyclictest66850irq/125-lan07:51:361
20227991515,0cyclictest66850irq/125-lan07:45:041
20227991515,0cyclictest66850irq/125-lan07:15:201
20227991414,0cyclictest66850irq/125-lan08:45:231
20227991313,0cyclictest66850irq/125-lan07:13:181
20227991212,0cyclictest66850irq/125-lan09:02:131
20227991212,0cyclictest66850irq/125-lan07:20:131
20227991111,0cyclictest66850irq/125-lan07:35:141
2023199107,0cyclictest191rcu_preempt10:48:242
2022199109,1cyclictest517-21systemd-resolve12:05:410
202369998,1cyclictest771-21snmpd08:35:303
202319998,1cyclictest771-21snmpd11:53:062
202319998,1cyclictest771-21snmpd09:38:532
202319998,1cyclictest771-21snmpd07:13:322
202319998,1cyclictest517-21systemd-resolve12:26:392
202319997,0cyclictest191rcu_preempt09:10:162
202319996,3cyclictest771-21snmpd07:30:012
202219998,1cyclictest769-21lldpd09:59:560
202219998,1cyclictest769-21lldpd09:11:310
202369988,0cyclictest771-21snmpd11:57:493
202369988,0cyclictest771-21snmpd11:50:243
202369988,0cyclictest771-21snmpd11:36:363
202369988,0cyclictest771-21snmpd11:28:353
202369988,0cyclictest771-21snmpd11:14:143
202369988,0cyclictest771-21snmpd08:45:243
202369988,0cyclictest771-21snmpd07:30:433
202369988,0cyclictest769-21lldpd12:04:243
202369988,0cyclictest769-21lldpd11:24:213
202369988,0cyclictest769-21lldpd10:49:263
202369988,0cyclictest769-21lldpd09:30:583
202369988,0cyclictest769-21lldpd09:23:373
202369988,0cyclictest769-21lldpd08:06:583
202369987,1cyclictest771-21snmpd11:15:143
202369987,1cyclictest771-21snmpd08:24:293
202369987,1cyclictest769-21lldpd08:30:123
202369987,1cyclictest769-21lldpd08:02:403
202369987,1cyclictest524-21avahi-daemon10:37:173
202369987,1cyclictest524-21avahi-daemon10:32:133
202369986,2cyclictest771-21snmpd10:06:203
202369986,2cyclictest771-21snmpd09:14:223
202319988,0cyclictest771-21snmpd09:18:422
202319988,0cyclictest771-21snmpd09:02:322
202319988,0cyclictest769-21lldpd11:00:572
202319988,0cyclictest524-21avahi-daemon10:12:532
202319987,1cyclictest771-21snmpd10:20:062
202319987,1cyclictest771-21snmpd08:13:082
202319987,1cyclictest771-21snmpd07:56:342
202319987,1cyclictest771-21snmpd07:54:102
202319986,2cyclictest771-21snmpd12:37:332
202319986,2cyclictest771-21snmpd12:00:522
202319986,2cyclictest771-21snmpd07:29:332
202319986,2cyclictest771-21snmpd07:17:512
202319986,2cyclictest749-21ntpd10:56:492
202319986,2cyclictest30290-21cat10:50:152
202319985,3cyclictest749-21ntpd08:20:372
202319985,1cyclictest191rcu_preempt11:10:202
202319985,1cyclictest191rcu_preempt10:35:162
202319984,2cyclictest18042-21systemd-run09:25:002
202319982,3cyclictest25535-21ntpq11:55:182
202219988,0cyclictest771-21snmpd11:54:220
202219988,0cyclictest771-21snmpd11:20:350
202219988,0cyclictest769-21lldpd11:34:550
202219988,0cyclictest769-21lldpd10:30:080
202219988,0cyclictest769-21lldpd10:13:140
202219988,0cyclictest769-21lldpd09:06:190
202219988,0cyclictest769-21lldpd07:19:170
202219987,1cyclictest771-21snmpd10:36:430
202219987,1cyclictest771-21snmpd08:15:080
202219987,1cyclictest771-21snmpd07:35:220
202219987,1cyclictest771-21snmpd07:20:180
202219987,1cyclictest524-21avahi-daemon12:39:070
202219986,2cyclictest771-21snmpd11:06:230
202219986,2cyclictest771-21snmpd11:00:550
202219986,2cyclictest771-21snmpd09:01:030
202219986,2cyclictest771-21snmpd08:24:500
202219986,2cyclictest771-21snmpd07:40:280
202369977,0cyclictest771-21snmpd11:33:123
202369977,0cyclictest771-21snmpd08:54:423
202369977,0cyclictest769-21lldpd11:45:573
202369976,1cyclictest771-21snmpd07:13:503
202369976,1cyclictest524-21avahi-daemon11:43:403
202369976,1cyclictest517-21systemd-resolve12:16:073
202369976,1cyclictest27171-21cat09:35:153
202369976,1cyclictest255-20systemd-journal09:59:073
202369976,1cyclictest12862-21cpuspeed_turbos08:10:133
202369975,2cyclictest771-21snmpd12:05:163
202369975,2cyclictest771-21snmpd11:04:223
202369975,2cyclictest771-21snmpd09:46:193
202369975,2cyclictest771-21snmpd08:41:033
202369974,3cyclictest550-21NetworkManager09:42:393
202369973,3cyclictest3877-21sed09:05:193
202369973,3cyclictest24487-21ssh10:44:293
202319977,0cyclictest771-21snmpd10:42:222
202319977,0cyclictest771-21snmpd10:25:092
202319977,0cyclictest771-21snmpd09:25:162
202319976,1cyclictest769-21lldpd12:07:162
202319976,1cyclictest769-21lldpd10:07:142
202319976,1cyclictest769-21lldpd09:31:442
202319976,1cyclictest749-21ntpd08:40:022
202319976,0cyclictest191rcu_preempt09:45:152
202319976,0cyclictest191rcu_preempt09:42:582
202319975,2cyclictest771-21snmpd09:05:272
202319975,1cyclictest771-21snmpd08:47:472
202319974,3cyclictest550-21NetworkManager11:25:022
202319974,2cyclictest255-20systemd-journal11:39:582
202319974,2cyclictest17493-21kworker/u16:1+xprtiod11:09:032
202319974,1cyclictest25474-21ntpq11:20:172
202319972,3cyclictest15518-21ntpq08:15:152
202319971,4cyclictest32367-21cat08:55:232
202319971,3cyclictest43-21ksoftirqd/210:04:542
202319971,2cyclictest0-21swapper/211:45:172
202319970,5cyclictest7961-21ntpq09:50:182
202319970,5cyclictest6455-21ntpq12:10:192
202319970,5cyclictest21950-21ntpq08:30:152
202319970,5cyclictest19164-21hddtemp_smartct08:25:142
202219977,0cyclictest771-21snmpd11:10:380
202219977,0cyclictest771-21snmpd07:34:100
202219977,0cyclictest769-21lldpd12:31:220
202219977,0cyclictest769-21lldpd12:14:300
202219977,0cyclictest769-21lldpd09:37:510
202219977,0cyclictest769-21lldpd08:35:550
202219977,0cyclictest524-21avahi-daemon07:56:230
202219976,1cyclictest771-21snmpd12:03:550
202219976,1cyclictest771-21snmpd09:24:550
202219976,1cyclictest524-21avahi-daemon11:15:100
202219976,1cyclictest524-21avahi-daemon09:29:040
202219975,2cyclictest771-21snmpd11:38:160
202219975,2cyclictest771-21snmpd11:25:530
202219975,2cyclictest771-21snmpd10:19:030
202219975,2cyclictest771-21snmpd08:04:280
202219975,2cyclictest7518-21cat09:50:150
202369966,0cyclictest771-21snmpd12:25:163
202369966,0cyclictest771-21snmpd11:08:563
202369966,0cyclictest769-21lldpd10:22:183
202369966,0cyclictest524-21avahi-daemon07:58:523
202369965,1cyclictest524-21avahi-daemon10:55:383
202369965,1cyclictest524-21avahi-daemon07:38:233
202369965,1cyclictest517-21systemd-resolve10:16:543
202369965,1cyclictest517-21systemd-resolve08:55:183
202369963,2cyclictest27147-21ssh12:31:373
202369963,2cyclictest20064-21ssh09:25:463
202369963,2cyclictest18849-21ssh10:01:563
202369962,3cyclictest32592-21ssh12:37:173
202369962,3cyclictest255-20systemd-journal10:12:533
202369962,3cyclictest23147-21ntpq07:15:163
202319966,0cyclictest771-21snmpd07:40:092
202319966,0cyclictest769-21lldpd11:41:132
202319966,0cyclictest421ktimers/210:30:152
202319964,2cyclictest20214-21cyclictest12:23:432
202319964,0cyclictest191rcu_preempt11:31:382
202319964,0cyclictest191rcu_preempt09:55:122
202319963,0cyclictest191rcu_preempt08:35:202
202319962,1cyclictest0-21swapper/208:05:212
202319961,3cyclictest25611-21ntpq12:30:162
202319961,3cyclictest25179-21ntpq07:20:162
202319961,2cyclictest10388-21hddtemp_smartct12:15:142
202319961,1cyclictest30514-21apt-get07:35:142
202319960,4cyclictest43-21ksoftirqd/210:19:202
202219966,0cyclictest769-21lldpd10:28:580
202219966,0cyclictest769-21lldpd08:05:300
202219965,1cyclictest771-21snmpd10:03:110
202219965,1cyclictest771-21snmpd09:34:220
202219965,1cyclictest771-21snmpd07:10:450
202219965,1cyclictest3277-21tail09:45:140
202219964,2cyclictest771-21snmpd09:43:290
202219964,2cyclictest771-21snmpd08:56:090
202219964,2cyclictest255-20systemd-journal10:20:570
202219963,3cyclictest3892-21sshd07:49:190
202219962,3cyclictest26121-21ntp_states08:40:190
202219962,3cyclictest18148-21ssh12:23:270
202369955,0cyclictest771-21snmpd09:00:173
202369955,0cyclictest771-21snmpd07:46:073
202369955,0cyclictest524-21avahi-daemon08:15:513
202369953,2cyclictest771-21snmpd12:11:163
202369953,2cyclictest771-21snmpd07:25:423
202369953,2cyclictest749-21ntpd07:20:523
202369953,2cyclictest15235-21cat12:20:163
202369952,3cyclictest550-21NetworkManager07:50:273
202369952,2cyclictest8008-21sed10:25:243
202369952,2cyclictest586-21rs:main3
202369952,2cyclictest20799-21latency_hist08:30:003
202369950,5cyclictest7933-21ntpq09:50:173
202369950,5cyclictest550-21NetworkManager10:52:523
202319952,1cyclictest191rcu_preempt08:00:152
202319951,3cyclictest43-21ksoftirqd/211:15:042
202319951,1cyclictest43-21ksoftirqd/208:50:162
202319950,3cyclictest2757-21cat07:45:152
202219955,0cyclictest771-21snmpd08:54:360
202219955,0cyclictest524-21avahi-daemon08:30:290
202219955,0cyclictest517-21systemd-resolve08:45:080
202219954,1cyclictest771-21snmpd07:51:490
202219954,1cyclictest584-21in:imuxsock10:43:190
202219952,3cyclictest550-21NetworkManager12:28:310
202219952,3cyclictest30771-21ssh10:50:200
202219952,2cyclictest586-21rs:main0
202219952,2cyclictest3496-21ssh10:55:450
202219952,2cyclictest26728-21ssh11:56:220
202219952,2cyclictest10029-21awk12:15:120
202219952,1cyclictest10506-21kworker/u16:4+rpciod11:41:390
202219951,3cyclictest26576-21sed10:45:220
202219950,5cyclictest0-21swapper/010:05:130
202219950,4cyclictest10220-21ntpq09:15:170
202369944,0cyclictest1038-21ntpq07:40:163
202219941,2cyclictest26579-21grep07:25:120
202219940,3cyclictest20537-21pluginstate08:30:000
202219940,3cyclictest12878-21df_inode08:10:140
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional