You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-27 - 03:03

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack8slot6s.osadl.org (updated Thu Nov 27, 2025 00:43:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
27331992626,0cyclictest4645-21fwupd20:56:191
27334991919,0cyclictest771-21snmpd23:17:472
27331991918,1cyclictest66850irq/125-lan23:45:171
27331991918,1cyclictest66850irq/125-lan23:30:131
27331991918,1cyclictest66850irq/125-lan23:06:281
27331991918,1cyclictest66850irq/125-lan22:30:571
27331991917,1cyclictest66850irq/125-lan21:50:121
27334991818,0cyclictest771-21snmpd23:36:022
27334991818,0cyclictest771-21snmpd23:22:072
27334991818,0cyclictest771-21snmpd23:04:372
27334991818,0cyclictest771-21snmpd22:44:132
27334991818,0cyclictest771-21snmpd21:39:322
27334991818,0cyclictest771-21snmpd20:46:412
27334991818,0cyclictest771-21snmpd20:05:562
27334991818,0cyclictest771-21snmpd19:45:332
27334991818,0cyclictest771-21snmpd00:28:232
27334991818,0cyclictest26314-21cat00:00:152
27331991818,0cyclictest66850irq/125-lan23:13:291
27331991818,0cyclictest66850irq/125-lan23:00:121
27331991818,0cyclictest66850irq/125-lan22:43:511
27331991818,0cyclictest66850irq/125-lan22:15:431
27331991818,0cyclictest66850irq/125-lan22:05:131
27331991818,0cyclictest66850irq/125-lan21:32:371
27331991818,0cyclictest66850irq/125-lan21:26:161
27331991818,0cyclictest66850irq/125-lan21:22:331
27331991818,0cyclictest66850irq/125-lan20:20:191
27331991818,0cyclictest66850irq/125-lan00:15:211
27331991817,1cyclictest66850irq/125-lan23:25:201
27331991817,1cyclictest66850irq/125-lan22:37:111
27331991817,1cyclictest66850irq/125-lan22:20:131
27331991817,1cyclictest66850irq/125-lan21:45:191
27331991817,1cyclictest66850irq/125-lan21:15:201
27331991817,1cyclictest66850irq/125-lan21:06:211
27331991817,1cyclictest66850irq/125-lan21:00:141
27331991817,1cyclictest66850irq/125-lan00:22:201
27331991817,1cyclictest66850irq/125-lan00:14:431
27334991717,0cyclictest771-21snmpd23:50:052
27334991717,0cyclictest771-21snmpd23:46:412
27334991717,0cyclictest771-21snmpd20:38:022
27331991717,0cyclictest66850irq/125-lan23:57:421
27331991717,0cyclictest66850irq/125-lan22:12:091
27331991717,0cyclictest66850irq/125-lan21:55:231
27331991717,0cyclictest66850irq/125-lan00:30:461
27331991717,0cyclictest66850irq/125-lan00:08:131
27331991716,1cyclictest66850irq/125-lan23:36:151
27331991716,1cyclictest66850irq/125-lan23:15:171
27331991716,1cyclictest66850irq/125-lan22:58:161
27331991716,1cyclictest66850irq/125-lan22:28:591
27331991716,1cyclictest66850irq/125-lan21:36:121
27331991716,1cyclictest66850irq/125-lan21:10:391
27331991716,1cyclictest66850irq/125-lan20:00:161
27331991716,1cyclictest66850irq/125-lan00:35:131
27331991716,1cyclictest66850irq/125-lan00:25:221
27331991616,0cyclictest66850irq/125-lan23:52:101
27331991616,0cyclictest66850irq/125-lan23:42:551
27331991616,0cyclictest66850irq/125-lan19:35:161
27331991615,1cyclictest66850irq/125-lan23:22:371
27331991615,1cyclictest66850irq/125-lan22:45:151
27331991615,1cyclictest66850irq/125-lan22:00:211
27331991615,1cyclictest66850irq/125-lan21:43:231
27331991615,1cyclictest66850irq/125-lan20:17:001
27331991615,1cyclictest66850irq/125-lan19:20:191
27331991615,1cyclictest66850irq/125-lan00:02:371
27331991514,1cyclictest66850irq/125-lan20:30:011
27331991514,1cyclictest66850irq/125-lan19:40:211
27331991414,0cyclictest66850irq/125-lan20:50:191
27331991414,0cyclictest66850irq/125-lan19:30:201
27331991414,0cyclictest66850irq/125-lan19:11:001
27331991413,1cyclictest66850irq/125-lan20:45:201
27331991413,1cyclictest66850irq/125-lan19:45:131
27331991413,1cyclictest66850irq/125-lan19:15:111
27331991212,0cyclictest66850irq/125-lan19:55:141
27331991211,1cyclictest66850irq/125-lan20:30:111
2733199109,1cyclictest66850irq/125-lan19:50:181
273379998,1cyclictest771-21snmpd21:27:423
273379998,1cyclictest771-21snmpd20:57:573
273379998,1cyclictest769-21lldpd22:02:123
273379998,1cyclictest524-21avahi-daemon22:13:543
273349998,1cyclictest771-21snmpd22:14:362
273349998,1cyclictest771-21snmpd20:03:522
273319999,0cyclictest66850irq/125-lan20:40:091
273319999,0cyclictest66850irq/125-lan20:14:201
273269998,1cyclictest771-21snmpd22:20:230
273269996,3cyclictest771-21snmpd23:47:270
273379988,0cyclictest771-21snmpd21:35:323
273379988,0cyclictest769-21lldpd20:32:133
273379988,0cyclictest769-21lldpd19:26:303
273379988,0cyclictest769-21lldpd00:25:053
273379987,1cyclictest769-21lldpd22:28:063
273379987,1cyclictest524-21avahi-daemon23:04:503
273379987,1cyclictest524-21avahi-daemon22:23:043
273379987,1cyclictest524-21avahi-daemon21:17:213
273379986,2cyclictest771-21snmpd20:38:423
273379986,2cyclictest771-21snmpd20:19:523
273379984,3cyclictest255-20systemd-journal23:11:093
273349988,0cyclictest771-21snmpd20:43:152
273349988,0cyclictest771-21snmpd20:13:082
273349988,0cyclictest769-21lldpd23:43:272
273349988,0cyclictest769-21lldpd20:57:562
273349988,0cyclictest769-21lldpd00:10:252
273349988,0cyclictest769-21lldpd00:07:482
273349987,1cyclictest771-21snmpd00:16:162
273349986,2cyclictest771-21snmpd21:16:362
273349986,2cyclictest749-21ntpd23:33:272
273349986,2cyclictest749-21ntpd23:27:352
273349986,2cyclictest749-21ntpd22:58:312
273349986,2cyclictest749-21ntpd22:50:082
273349986,2cyclictest749-21ntpd22:47:082
273349986,2cyclictest749-21ntpd22:36:082
273349986,2cyclictest749-21ntpd22:32:392
273349986,2cyclictest749-21ntpd22:21:362
273349986,2cyclictest749-21ntpd22:05:282
273349986,2cyclictest749-21ntpd22:00:012
273349986,2cyclictest749-21ntpd21:50:062
273349986,2cyclictest749-21ntpd21:45:332
273349986,2cyclictest749-21ntpd21:43:322
273349986,2cyclictest749-21ntpd21:26:132
273349986,2cyclictest749-21ntpd21:22:572
273349986,2cyclictest749-21ntpd21:12:172
273349986,2cyclictest749-21ntpd00:35:092
273349986,2cyclictest749-21ntpd00:34:142
273349986,2cyclictest749-21ntpd00:24:292
273319988,0cyclictest66850irq/125-lan20:35:021
273269988,0cyclictest769-21lldpd23:56:350
273269988,0cyclictest769-21lldpd23:51:280
273269988,0cyclictest769-21lldpd22:06:360
273269987,1cyclictest771-21snmpd21:45:310
273269987,1cyclictest771-21snmpd20:04:040
273269987,1cyclictest771-21snmpd00:05:360
273269987,1cyclictest769-21lldpd22:29:120
273269987,1cyclictest584-21in:imuxsock00:33:400
273269987,1cyclictest517-21systemd-resolve00:23:010
273269986,2cyclictest771-21snmpd22:04:200
273269986,2cyclictest771-21snmpd21:42:470
273269986,2cyclictest771-21snmpd21:08:300
273269986,2cyclictest771-21snmpd19:30:030
273269986,2cyclictest6110-21cat21:00:130
273379977,0cyclictest771-21snmpd23:25:573
273379977,0cyclictest771-21snmpd20:51:363
273379977,0cyclictest771-21snmpd20:41:323
273379977,0cyclictest771-21snmpd00:11:103
273379977,0cyclictest769-21lldpd23:37:563
273379977,0cyclictest769-21lldpd22:47:513
273379977,0cyclictest769-21lldpd22:16:303
273379977,0cyclictest769-21lldpd21:45:103
273379977,0cyclictest769-21lldpd19:30:163
273379977,0cyclictest769-21lldpd00:03:483
273379976,1cyclictest769-21lldpd23:07:403
273379976,1cyclictest584-21in:imuxsock22:44:453
273379976,1cyclictest524-21avahi-daemon21:20:203
273379976,1cyclictest27588-21cpuspeed_turbos19:10:113
273379976,1cyclictest16423-21ntp_kernel_pll_23:50:203
273379975,2cyclictest771-21snmpd22:57:233
273379975,2cyclictest771-21snmpd19:51:143
273379975,2cyclictest771-21snmpd00:38:233
273379975,2cyclictest771-21snmpd00:22:463
273379973,4cyclictest5935-21uniq21:00:123
273349977,0cyclictest771-21snmpd20:54:092
273349977,0cyclictest769-21lldpd19:37:172
273349976,1cyclictest771-21snmpd19:20:102
273349976,1cyclictest749-21ntpd23:59:282
273349976,1cyclictest749-21ntpd22:18:432
273349976,1cyclictest749-21ntpd19:10:002
273349976,1cyclictest524-21avahi-daemon23:12:052
273349976,1cyclictest30613-21needreboot19:15:142
273349975,2cyclictest771-21snmpd20:29:452
273349975,2cyclictest749-21ntpd23:05:082
273349975,2cyclictest749-21ntpd21:30:382
273319976,1cyclictest66850irq/125-lan20:05:141
273269977,0cyclictest771-21snmpd21:57:510
273269977,0cyclictest771-21snmpd20:21:220
273269977,0cyclictest771-21snmpd20:10:230
273269977,0cyclictest769-21lldpd22:46:340
273269977,0cyclictest769-21lldpd22:41:440
273269977,0cyclictest769-21lldpd21:37:260
273269977,0cyclictest769-21lldpd19:56:350
273269977,0cyclictest769-21lldpd19:44:570
273269976,1cyclictest32351-21gltestperf19:20:160
273269976,1cyclictest255-20systemd-journal23:32:220
273269976,1cyclictest1475-21gltestperf20:50:120
273269976,1cyclictest12304-21needreboot19:50:150
273269975,2cyclictest771-21snmpd21:25:280
273269975,2cyclictest771-21snmpd19:35:180
273269975,2cyclictest771-21snmpd00:18:520
273269974,2cyclictest255-20systemd-journal21:31:050
273269973,3cyclictest8599-21ssh23:11:360
273379966,0cyclictest771-21snmpd19:49:283
273379965,1cyclictest771-21snmpd23:49:353
273379965,1cyclictest31503-21fschecks_time00:05:123
273379964,2cyclictest771-21snmpd23:23:373
273379963,3cyclictest9339-21perl21:40:123
273379963,3cyclictest550-21NetworkManager22:50:133
273379963,3cyclictest550-21NetworkManager21:59:143
273379963,2cyclictest255-20systemd-journal23:30:453
273379963,2cyclictest255-20systemd-journal22:06:543
273379962,3cyclictest11263-21sh00:17:013
273379961,5cyclictest550-21NetworkManager21:06:473
273379961,5cyclictest550-21NetworkManager00:30:333
273349966,0cyclictest769-21lldpd20:21:422
273349966,0cyclictest23075-21unixbench_singl20:15:222
273349965,1cyclictest771-21snmpd19:28:482
273349965,1cyclictest12130-21cat19:50:152
273349964,2cyclictest771-21snmpd19:42:222
273349964,2cyclictest749-21ntpd22:25:122
273349964,2cyclictest749-21ntpd21:59:182
273319966,0cyclictest3367-21idleruntime-cro19:30:011
273269966,0cyclictest769-21lldpd23:02:160
273269966,0cyclictest769-21lldpd22:59:480
273269966,0cyclictest769-21lldpd21:11:120
273269966,0cyclictest769-21lldpd19:25:530
273269966,0cyclictest769-21lldpd00:35:020
273269965,1cyclictest771-21snmpd20:06:340
273269965,1cyclictest771-21snmpd00:29:140
273269965,1cyclictest771-21snmpd00:14:220
273269965,1cyclictest769-21lldpd23:26:540
273269964,2cyclictest771-21snmpd22:52:570
273269964,2cyclictest771-21snmpd20:17:580
273269963,3cyclictest550-21NetworkManager23:36:010
273269963,2cyclictest876-21ssh22:34:020
273269963,2cyclictest586-21rs:main0
273269963,2cyclictest255-20systemd-journal00:01:160
273269962,3cyclictest1995-21sed23:05:190
273269962,3cyclictest19423-21ssh23:22:270
273379955,0cyclictest771-21snmpd22:35:283
273379954,1cyclictest29252-21cpuspeed_turbos22:30:143
273379953,2cyclictest586-21rs:main3
273379952,3cyclictest550-21NetworkManager21:30:493
273379952,3cyclictest12058-21cut23:15:143
273379952,2cyclictest586-21rs:main3
273379952,2cyclictest32416-21sh20:45:173
273379952,2cyclictest255-20systemd-journal19:36:323
273379952,2cyclictest24946-21sort20:20:193
273379952,2cyclictest20551-21ssh23:55:013
273379952,2cyclictest13983-21tune2fs19:55:153
273379951,3cyclictest18519-21ntpq20:05:173
273379950,5cyclictest20258-21ntpq21:50:163
273349955,0cyclictest769-21lldpd19:56:202
273349954,1cyclictest749-21ntpd21:05:132
273349954,1cyclictest6518-21ntp_states21:00:172
273349954,1cyclictest0-21swapper/219:30:112
273269955,0cyclictest771-21snmpd19:46:030
273269955,0cyclictest769-21lldpd20:40:220
273269955,0cyclictest19569-21cpuspeed_turbos21:50:110
273269954,1cyclictest771-21snmpd19:10:240
273269954,1cyclictest31860-21gltestperf20:45:130
273269954,1cyclictest171ktimers/023:15:340
273269954,1cyclictest14322-21ntp_states22:15:150
273269953,2cyclictest771-21snmpd20:27:570
273269952,2cyclictest9974-21ssh22:10:500
273269952,2cyclictest3809-21ssh22:35:500
273269952,2cyclictest16798-21ssh21:15:210
273269951,3cyclictest5474-21perl23:40:110
273379943,1cyclictest511ktimers/319:40:013
273379942,2cyclictest5564-21tail23:40:123
273379941,2cyclictest29854-21awk19:15:003
273379941,2cyclictest25894-21seq20:26:253
273379940,3cyclictest31987-21apt-get19:20:143
273379940,3cyclictest15888-21sh20:00:143
273349941,2cyclictest26692-21dpkg20:30:002
273269941,2cyclictest30704-21ld19:15:150
273269941,2cyclictest28854-21cut20:35:000
273269940,4cyclictest3922-21ntpq20:55:160
273269930,3cyclictest28907-21chrt20:35:370
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional