You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-01 - 08:08

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack8slot6s.osadl.org (updated Tue Jul 01, 2025 00:43:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
27362993434,0cyclictest5787-21fwupd00:28:180
27362991918,1cyclictest66850irq/125-lan23:25:160
27362991918,1cyclictest66850irq/125-lan23:01:410
27362991918,1cyclictest66850irq/125-lan21:48:200
27362991918,1cyclictest66850irq/125-lan19:13:330
27371991818,0cyclictest771-21snmpd23:39:042
27371991818,0cyclictest771-21snmpd23:21:062
27371991818,0cyclictest771-21snmpd22:13:072
27371991818,0cyclictest771-21snmpd21:25:422
27371991818,0cyclictest771-21snmpd19:27:382
27371991818,0cyclictest30562-21cat00:20:172
27371991818,0cyclictest15377-21cat22:55:182
27362991818,0cyclictest66850irq/125-lan22:34:450
27362991818,0cyclictest66850irq/125-lan21:30:200
27362991818,0cyclictest66850irq/125-lan21:26:190
27362991818,0cyclictest66850irq/125-lan21:20:200
27362991818,0cyclictest66850irq/125-lan21:15:160
27362991818,0cyclictest66850irq/125-lan20:20:330
27362991817,1cyclictest66850irq/125-lan23:54:420
27362991817,1cyclictest66850irq/125-lan23:40:160
27362991817,1cyclictest66850irq/125-lan23:31:110
27362991817,1cyclictest66850irq/125-lan23:18:110
27362991817,1cyclictest66850irq/125-lan23:05:210
27362991817,1cyclictest66850irq/125-lan22:05:200
27362991817,1cyclictest66850irq/125-lan22:04:200
27362991817,1cyclictest66850irq/125-lan21:37:090
27362991817,1cyclictest66850irq/125-lan20:45:000
27362991817,1cyclictest66850irq/125-lan19:57:250
27362991817,1cyclictest66850irq/125-lan19:40:250
27362991817,1cyclictest66850irq/125-lan19:32:320
27362991817,1cyclictest66850irq/125-lan00:31:250
27362991817,1cyclictest66850irq/125-lan00:23:580
27362991817,1cyclictest66850irq/125-lan00:17:490
27362991817,1cyclictest66850irq/125-lan00:06:580
27371991717,0cyclictest771-21snmpd21:13:292
27371991717,0cyclictest771-21snmpd20:44:142
27371991717,0cyclictest771-21snmpd19:34:352
27362991717,0cyclictest66850irq/125-lan23:56:550
27362991717,0cyclictest66850irq/125-lan23:20:280
27362991717,0cyclictest66850irq/125-lan23:10:240
27362991717,0cyclictest66850irq/125-lan22:25:150
27362991717,0cyclictest66850irq/125-lan21:41:120
27362991716,1cyclictest66850irq/125-lan23:45:370
27362991716,1cyclictest66850irq/125-lan23:35:130
27362991716,1cyclictest66850irq/125-lan22:50:130
27362991716,1cyclictest66850irq/125-lan22:45:580
27362991716,1cyclictest66850irq/125-lan22:44:060
27362991716,1cyclictest66850irq/125-lan22:39:310
27362991716,1cyclictest66850irq/125-lan22:20:030
27362991716,1cyclictest66850irq/125-lan21:52:560
27362991716,1cyclictest66850irq/125-lan21:10:180
27362991716,1cyclictest66850irq/125-lan00:10:130
27362991716,1cyclictest66850irq/125-lan00:01:300
27371991616,0cyclictest771-21snmpd22:26:122
27362991616,0cyclictest66850irq/125-lan22:56:410
27362991616,0cyclictest66850irq/125-lan22:10:230
27362991616,0cyclictest66850irq/125-lan20:56:570
27362991616,0cyclictest66850irq/125-lan20:50:150
27362991616,0cyclictest66850irq/125-lan20:28:290
27362991616,0cyclictest66850irq/125-lan19:45:020
27362991615,1cyclictest66850irq/125-lan22:18:150
27362991615,1cyclictest66850irq/125-lan21:57:070
27371991515,0cyclictest771-21snmpd23:31:402
27371991515,0cyclictest771-21snmpd19:21:212
27362991515,0cyclictest66850irq/125-lan00:39:030
27362991514,1cyclictest66850irq/125-lan20:30:200
27362991514,1cyclictest66850irq/125-lan19:50:130
27362991514,1cyclictest66850irq/125-lan19:39:330
27371991413,1cyclictest749-21ntpd21:24:522
27362991414,0cyclictest66850irq/125-lan20:00:170
27362991414,0cyclictest66850irq/125-lan19:26:500
27362991413,1cyclictest66850irq/125-lan20:45:170
27362991413,1cyclictest66850irq/125-lan20:05:170
27366991311,2cyclictest771-21snmpd23:17:061
27362991313,0cyclictest66850irq/125-lan21:00:140
27362991313,0cyclictest66850irq/125-lan19:22:020
27362991312,1cyclictest66850irq/125-lan20:10:150
2737399128,4cyclictest524-21avahi-daemon23:01:473
2737399127,1cyclictest13311-21sh22:53:023
2736699120,11cyclictest22044-21taskset22:28:161
2737399118,1cyclictest771-21snmpd21:55:133
27362991111,0cyclictest66850irq/125-lan20:15:240
2737399108,0cyclictest769-21lldpd21:29:213
2736699109,1cyclictest517-21systemd-resolve00:35:211
27366991010,0cyclictest771-21snmpd23:54:551
27366991010,0cyclictest771-21snmpd21:29:361
27366991010,0cyclictest771-21snmpd20:53:211
27362991010,0cyclictest66850irq/125-lan21:05:170
273739998,1cyclictest771-21snmpd22:08:203
273739998,1cyclictest771-21snmpd20:45:113
273739998,1cyclictest517-21systemd-resolve22:46:473
273739997,0cyclictest771-21snmpd21:03:383
273719998,1cyclictest771-21snmpd21:15:392
273719996,2cyclictest12044-21grep19:50:152
273669999,0cyclictest6609-21cat23:20:181
273669998,1cyclictest771-21snmpd20:19:491
273669998,0cyclictest771-21snmpd21:21:071
273629999,0cyclictest66850irq/125-lan20:35:200
273629999,0cyclictest66850irq/125-lan20:35:190
273629998,1cyclictest66850irq/125-lan19:15:170
273739988,0cyclictest771-21snmpd23:41:533
273739988,0cyclictest771-21snmpd22:42:143
273739988,0cyclictest769-21lldpd22:55:533
273739987,1cyclictest771-21snmpd22:34:383
273739987,1cyclictest771-21snmpd22:28:443
273739987,1cyclictest771-21snmpd20:25:253
273739987,1cyclictest517-21systemd-resolve22:17:053
273739986,2cyclictest771-21snmpd23:11:043
273739986,2cyclictest771-21snmpd21:16:243
273739986,2cyclictest771-21snmpd19:48:483
273739986,2cyclictest771-21snmpd00:24:023
273739986,2cyclictest749-21ntpd20:22:403
273739986,2cyclictest749-21ntpd20:07:113
273739986,2cyclictest749-21ntpd20:00:103
273739986,2cyclictest749-21ntpd19:51:513
273739986,2cyclictest749-21ntpd19:21:153
273719988,0cyclictest771-21snmpd23:49:522
273719988,0cyclictest771-21snmpd22:45:532
273719988,0cyclictest769-21lldpd22:30:102
273719987,1cyclictest524-21avahi-daemon20:21:432
273719986,2cyclictest771-21snmpd00:39:352
273719986,2cyclictest749-21ntpd23:52:442
273719986,2cyclictest749-21ntpd23:19:412
273719986,2cyclictest749-21ntpd23:06:132
273719986,2cyclictest749-21ntpd23:02:012
273719986,2cyclictest749-21ntpd22:36:332
273719986,2cyclictest749-21ntpd21:42:082
273719986,2cyclictest749-21ntpd21:35:392
273719986,2cyclictest749-21ntpd21:30:012
273719986,2cyclictest749-21ntpd00:33:452
273719986,2cyclictest749-21ntpd00:29:012
273719986,2cyclictest749-21ntpd00:03:402
273669988,0cyclictest769-21lldpd23:46:071
273669988,0cyclictest769-21lldpd21:02:351
273669987,1cyclictest771-21snmpd20:25:321
273669986,2cyclictest771-21snmpd21:57:351
273669986,2cyclictest771-21snmpd20:55:051
273669986,2cyclictest771-21snmpd00:32:091
273739977,0cyclictest771-21snmpd23:37:043
273739977,0cyclictest771-21snmpd23:26:463
273739977,0cyclictest771-21snmpd22:02:013
273739977,0cyclictest769-21lldpd00:25:293
273739976,1cyclictest771-21snmpd23:53:193
273739976,1cyclictest771-21snmpd20:18:583
273739976,1cyclictest769-21lldpd22:36:473
273739976,1cyclictest7526-21cat23:55:163
273739976,1cyclictest524-21avahi-daemon21:11:203
273739976,1cyclictest524-21avahi-daemon00:16:273
273739975,2cyclictest749-21ntpd19:55:103
273739973,3cyclictest6606-21diskstats20:55:123
273739973,3cyclictest12999-21ssh00:35:273
273719977,0cyclictest771-21snmpd23:40:052
273719977,0cyclictest771-21snmpd22:40:032
273719977,0cyclictest771-21snmpd22:02:042
273719977,0cyclictest769-21lldpd23:12:282
273719976,1cyclictest771-21snmpd19:37:552
273719976,1cyclictest769-21lldpd20:05:202
273719976,1cyclictest749-21ntpd23:26:182
273719976,1cyclictest749-21ntpd22:51:352
273719976,1cyclictest749-21ntpd19:55:202
273719976,1cyclictest29981-21cpuspeed_turbos20:35:142
273719976,1cyclictest29981-21cpuspeed_turbos20:35:132
273719976,1cyclictest20073-21cpuspeed_turbos00:10:092
273719975,2cyclictest771-21snmpd22:21:242
273719972,4cyclictest255-20systemd-journal20:00:392
273669977,0cyclictest771-21snmpd23:11:431
273669977,0cyclictest771-21snmpd22:47:291
273669977,0cyclictest771-21snmpd22:22:111
273669977,0cyclictest771-21snmpd22:18:281
273669977,0cyclictest771-21snmpd21:30:421
273669977,0cyclictest771-21snmpd20:10:321
273669977,0cyclictest771-21snmpd19:38:451
273669977,0cyclictest771-21snmpd19:27:081
273669977,0cyclictest771-21snmpd00:16:331
273669977,0cyclictest769-21lldpd22:08:281
273669976,1cyclictest771-21snmpd21:08:001
273669976,1cyclictest771-21snmpd19:47:061
273669976,1cyclictest524-21avahi-daemon23:25:531
273669976,1cyclictest31538-21unixbench_singl00:20:231
273669975,2cyclictest771-21snmpd20:30:071
273669975,2cyclictest771-21snmpd00:12:181
273669975,2cyclictest749-21ntpd22:55:501
273669975,2cyclictest749-21ntpd22:34:091
273739966,0cyclictest524-21avahi-daemon21:33:023
273739966,0cyclictest524-21avahi-daemon19:41:223
273739965,1cyclictest771-21snmpd21:50:563
273739965,1cyclictest524-21avahi-daemon20:32:213
273739965,1cyclictest524-21avahi-daemon19:15:023
273739964,2cyclictest771-21snmpd23:19:423
273739964,2cyclictest771-21snmpd21:20:073
273739964,2cyclictest771-21snmpd00:11:273
273739963,3cyclictest550-21NetworkManager22:12:333
273739963,3cyclictest27844-21perl19:10:143
273739963,3cyclictest255-20systemd-journal00:01:583
273739963,3cyclictest17073-21ssh00:05:503
273739963,2cyclictest6059-21gpg-connect-age20:55:003
273739962,3cyclictest24166-21ssh23:05:083
273739962,3cyclictest15696-21ssh21:47:043
273739962,3cyclictest12086-21ssh21:43:473
273739961,5cyclictest550-21NetworkManager21:35:573
273719966,0cyclictest771-21snmpd22:15:312
273719966,0cyclictest771-21snmpd21:47:552
273719966,0cyclictest769-21lldpd22:06:012
273719966,0cyclictest769-21lldpd19:49:322
273719965,1cyclictest771-21snmpd21:02:022
273719965,1cyclictest749-21ntpd23:55:412
273719965,1cyclictest749-21ntpd00:16:352
273719965,1cyclictest524-21avahi-daemon19:40:062
273719964,2cyclictest749-21ntpd21:05:242
273669966,0cyclictest771-21snmpd23:35:431
273669966,0cyclictest771-21snmpd23:04:301
273669966,0cyclictest771-21snmpd20:40:441
273669966,0cyclictest771-21snmpd19:58:481
273669966,0cyclictest769-21lldpd23:09:211
273669966,0cyclictest769-21lldpd21:19:471
273669965,1cyclictest771-21snmpd22:41:271
273669965,1cyclictest524-21avahi-daemon23:43:571
273669965,1cyclictest517-21systemd-resolve21:35:301
273669965,1cyclictest255-20systemd-journal00:06:121
273669965,1cyclictest12911-21proc_pri00:00:191
273669964,2cyclictest771-21snmpd23:30:091
273669964,2cyclictest550-21NetworkManager21:11:361
273669963,3cyclictest550-21NetworkManager23:55:591
273669963,3cyclictest16370-21sed20:00:151
273669963,2cyclictest8851-21fw_forwarded_lo21:40:141
273669963,2cyclictest6333-21ssh22:10:551
273669963,2cyclictest21911-21ssh21:54:111
273669962,3cyclictest30416-21ntp_states19:15:161
273669962,3cyclictest13899-21ssh22:53:591
273739955,0cyclictest524-21avahi-daemon19:33:183
273739954,1cyclictest771-21snmpd20:44:353
273739953,2cyclictest550-21NetworkManager21:07:343
273739952,3cyclictest2110-21open_files19:25:193
273739952,3cyclictest14125-21apt-get22:20:173
273739952,2cyclictest7646-21ssh23:20:313
273739952,2cyclictest6496-21unixbench-2d19:35:283
273739952,2cyclictest31201-21ssh23:45:493
273739952,2cyclictest18801-21ssh23:33:153
273719954,1cyclictest0-21swapper/221:50:192
273719953,2cyclictest771-21snmpd20:32:012
273719953,2cyclictest771-21snmpd00:05:562
273719953,2cyclictest749-21ntpd20:10:062
273719952,3cyclictest26431-21memory20:25:172
273719952,2cyclictest255-20systemd-journal20:55:142
273719952,2cyclictest22751-21sed20:15:222
273719952,2cyclictest2241-21tune2fs20:45:142
273719951,3cyclictest30724-21sed19:15:192
273719950,3cyclictest23731-21ssh21:55:202
273669955,0cyclictest32059-21cat19:20:211
273669955,0cyclictest3029-21cat00:25:151
273669954,1cyclictest771-21snmpd21:45:031
273669954,1cyclictest771-21snmpd19:40:541
273669953,2cyclictest749-21ntpd20:38:221
273669953,2cyclictest749-21ntpd20:38:211
273669953,2cyclictest749-21ntpd19:11:311
273669952,3cyclictest24240-21sed20:20:181
273669952,2cyclictest3559-21seq20:47:591
273669952,2cyclictest29162-21ssh22:35:211
273669950,4cyclictest27849-21diskstats22:00:121
273669950,4cyclictest12493-21ntpq19:50:171
273739943,1cyclictest20198-21hddtemp_smartct20:10:163
273739941,2cyclictest749-21ntpd20:35:183
273739941,2cyclictest749-21ntpd20:35:183
273739940,4cyclictest7941-21ntpq00:30:183
273719942,2cyclictest749-21ntpd20:50:082
273719942,2cyclictest28434-21nvmesmart_nvme019:10:192
273669941,2cyclictest17976-21fschecks_count20:05:121
273669940,1cyclictest38601needreboot19:30:171
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional