You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-03 - 07:15

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack8slot6s.osadl.org (updated Thu Sep 03, 2026 00:43:39)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
71859910481047,1cyclictest30392-21kworker/1:2+events00:04:521
71949910471046,1cyclictest30023-21kworker/3:0+events20:06:153
71819910471045,2cyclictest14029-21kworker/0:2+events23:16:390
71899910441040,2cyclictest14515-21kworker/2:1+events22:33:272
71899910431041,2cyclictest11898-21kworker/2:0+events20:40:452
71859910421041,1cyclictest15744-21kworker/1:0+events00:12:401
71859910421040,2cyclictest7899-21turbostat20:34:551
71899910381036,2cyclictest20104-21kworker/2:1+events19:17:332
71949910371035,2cyclictest2846-21turbostat23:59:553
71949910361034,2cyclictest20452-21kworker/3:2+events21:41:063
71899910361034,2cyclictest1282-21kworker/2:0+events22:37:222
71899910351033,2cyclictest14515-21kworker/2:1+events23:36:502
71899910351033,2cyclictest14515-21kworker/2:1+events22:27:322
71949910331031,2cyclictest21599-21kworker/3:1+events22:51:053
71949910331031,2cyclictest21599-21kworker/3:1+events22:51:053
71899910321029,3cyclictest27218-21kworker/2:1+events23:47:522
71949910301029,1cyclictest2465-21kworker/3:1+events21:28:383
71949910301027,3cyclictest27680-21turbostat22:19:553
71899910301028,2cyclictest14515-21kworker/2:1+events22:20:222
71949910291027,2cyclictest14882-21kworker/3:0+events21:49:253
71899910291027,2cyclictest4914-21kworker/2:0+events19:12:342
71899910281024,3cyclictest11898-21kworker/2:0+events20:48:382
71949910271025,2cyclictest29207-21kworker/3:0+events21:16:283
71949910271025,2cyclictest21669-21kworker/3:1+events21:02:533
71899910231018,4cyclictest14402-21kworker/2:2+events23:42:452
71949910221021,1cyclictest2465-21kworker/3:1+events21:32:363
71949910221020,2cyclictest32068-21kworker/3:1+events20:10:443
71949910221020,2cyclictest10069-21kworker/3:2+events22:13:293
71899910221019,3cyclictest9381-21kworker/2:2+events19:22:102
71859910221021,1cyclictest29619-21kworker/1:1+events00:34:131
71899910211017,2cyclictest11898-21kworker/2:0+events20:50:222
71949910201019,1cyclictest19723-21kworker/3:2+events19:37:343
71949910201019,1cyclictest14882-21kworker/3:0+events22:06:303
71949910201018,2cyclictest14882-21kworker/3:0+events21:52:333
71949910201017,3cyclictest21599-21kworker/3:1+events22:48:183
71949910181017,1cyclictest7783-21kworker/3:0+events20:57:363
71949910181017,1cyclictest32068-21kworker/3:1+events20:28:393
71949910181017,1cyclictest21669-21kworker/3:1+events21:09:353
71949910181017,1cyclictest14882-21kworker/3:0+events22:00:233
71949910181015,3cyclictest14882-21kworker/3:0+events21:39:043
71949910171016,1cyclictest27949-21kworker/3:2+events20:00:323
71949910171015,2cyclictest5724-21kworker/3:2+events20:33:373
71949910161015,1cyclictest25901-21kworker/3:1+events19:59:433
71949910141010,2cyclictest10069-21kworker/3:2+events22:41:513
71819910141012,2cyclictest1812-21turbostat20:19:560
71949910121009,3cyclictest2465-21kworker/3:1+events21:20:433
71949910111009,2cyclictest27949-21kworker/3:2+events20:15:223
71859910111010,1cyclictest3486-21kworker/1:2+events00:28:261
71949910091007,2cyclictest4576-21kworker/3:1+events21:56:063
71949910091007,2cyclictest3654-21kworker/3:0+events20:22:413
71859910081007,1cyclictest15744-21kworker/1:0+events00:22:331
71949910061004,2cyclictest23847-21kworker/3:2+events19:50:203
71859910061005,1cyclictest24025-21kworker/1:1+events23:00:231
71819910061005,1cyclictest31926-21kworker/0:1+events19:29:370
71949910031001,2cyclictest8847-21turbostat23:34:553
71859910031002,1cyclictest22829-21kworker/1:1+events23:32:531
71859910031002,1cyclictest17253-21kworker/1:0+events23:14:011
71899910021000,2cyclictest21682-21turbostat22:44:562
71859910021001,1cyclictest15588-21kworker/1:2+events19:31:271
71859910021000,2cyclictest30392-21kworker/1:2+events23:55:211
7194991001998,2cyclictest21788-21kworker/3:0+events19:42:213
7185991000999,1cyclictest22829-21kworker/1:1+events23:21:061
718599999998,1cyclictest31205-21kworker/1:2+events23:29:331
719499998997,1cyclictest7783-21kworker/3:0+events20:37:103
719499998996,2cyclictest20462-21turbostat23:14:553
719499997996,1cyclictest21788-21kworker/3:0+events19:46:513
718599997996,1cyclictest4790-21kworker/1:0+events22:57:111
718599996995,1cyclictest17253-21kworker/1:0+events23:07:231
718599995994,1cyclictest30392-21kworker/1:2+events23:50:421
718599992991,1cyclictest30392-21kworker/1:2+events00:17:021
718599992991,1cyclictest30392-21kworker/1:2+events00:06:241
718599992991,1cyclictest15588-21kworker/1:2+events21:11:051
718599991989,2cyclictest14276-21kworker/1:0+events00:39:241
719499969966,3cyclictest11134-21turbostat19:24:543
718999945942,3cyclictest24044-21turbostat21:14:562
718599939938,1cyclictest22100-21turbostat19:49:541
718999938934,4cyclictest24114-21turbostat19:54:552
718599937934,3cyclictest9910-21turbostat20:39:561
718599933931,2cyclictest7298-21turbostat21:29:561
718599922918,3cyclictest32073-21turbostat22:54:561
718599922918,3cyclictest32073-21turbostat22:54:551
719499904901,3cyclictest11921-21turbostat20:44:553
718199901898,3cyclictest15969-21turbostat20:54:560
718199896894,2cyclictest16497-21turbostat22:39:550
718199884882,2cyclictest15607-21turbostat19:34:540
718599875873,2cyclictest19997-21turbostat21:04:551
718599870869,1cyclictest13592-21turbostat19:29:541
718599864861,3cyclictest17624-21turbostat19:39:551
718599686683,3cyclictest22032-21turbostat21:09:551
718199631628,3cyclictest20462-21turbostat23:14:550
718999603600,3cyclictest27680-21turbostat22:19:552
719499577576,1cyclictest15969-21turbostat20:54:563
718999575573,2cyclictest8847-21turbostat23:34:552
719499574572,2cyclictest16497-21turbostat22:39:553
718599571567,4cyclictest25660-21turbostat23:19:561
719499562560,2cyclictest15607-21turbostat19:34:543
718599542540,2cyclictest24855-21turbostat23:49:541
718999521519,2cyclictest9910-21turbostat20:39:562
718199507504,3cyclictest11134-21turbostat19:24:550
719499481479,2cyclictest24044-21turbostat21:14:563
7181994250,424cyclictest6966-21ssh22:55:420
7194992119,2cyclictest67750irq/123-lan00:20:123
7194991919,0cyclictest67750irq/123-lan22:25:133
7194991918,1cyclictest67750irq/123-lan23:05:163
7194991918,1cyclictest67750irq/123-lan00:05:163
7194991917,2cyclictest67750irq/123-lan00:35:173
7194991917,2cyclictest67750irq/123-lan00:32:283
7194991818,0cyclictest67750irq/123-lan23:50:213
7194991818,0cyclictest67750irq/123-lan00:13:413
7194991817,1cyclictest67750irq/123-lan23:40:173
7194991817,1cyclictest67750irq/123-lan23:20:233
7194991817,1cyclictest67750irq/123-lan00:26:093
7194991817,1cyclictest67750irq/123-lan00:19:403
7194991816,2cyclictest67750irq/123-lan23:04:143
7194991816,2cyclictest67750irq/123-lan22:23:273
7194991816,2cyclictest67750irq/123-lan00:02:513
7185991818,0cyclictest784-21snmpd22:16:311
7185991818,0cyclictest784-21snmpd22:14:501
7194991717,0cyclictest67750irq/123-lan23:25:143
7194991717,0cyclictest67750irq/123-lan19:19:383
7194991715,2cyclictest67750irq/123-lan23:47:593
7185991717,0cyclictest14893-21cat23:35:151
7194991616,0cyclictest67750irq/123-lan22:55:343
7194991615,1cyclictest67750irq/123-lan23:38:173
7194991615,1cyclictest67750irq/123-lan23:17:123
7194991615,1cyclictest67750irq/123-lan22:30:133
7194991614,2cyclictest67750irq/123-lan20:45:183
7194991514,1cyclictest67750irq/123-lan19:25:153
71899998,1cyclictest784-21snmpd21:00:592
71899998,1cyclictest784-21snmpd19:38:242
71859998,1cyclictest784-21snmpd21:52:041
71859998,1cyclictest784-21snmpd20:19:311
71859998,1cyclictest784-21snmpd19:42:531
71819998,1cyclictest784-21snmpd20:41:010
71819998,1cyclictest784-21snmpd20:37:440
71819998,1cyclictest782-21lldpd21:52:530
71899988,0cyclictest782-21lldpd20:19:542
71899988,0cyclictest782-21lldpd19:43:522
71899987,1cyclictest784-21snmpd20:58:592
71899987,1cyclictest782-21lldpd22:51:332
71899987,1cyclictest782-21lldpd22:51:332
71899987,1cyclictest526-21avahi-daemon23:07:162
71899986,2cyclictest784-21snmpd23:56:422
71899986,2cyclictest784-21snmpd21:55:372
71899986,2cyclictest784-21snmpd21:30:162
71899986,2cyclictest784-21snmpd20:13:272
71899986,2cyclictest784-21snmpd00:09:112
71859988,0cyclictest782-21lldpd22:31:331
71859988,0cyclictest782-21lldpd21:15:091
71859988,0cyclictest782-21lldpd19:52:541
71859988,0cyclictest782-21lldpd19:10:091
71859987,1cyclictest784-21snmpd20:22:571
71859987,1cyclictest526-21avahi-daemon21:32:181
71859986,2cyclictest784-21snmpd22:03:371
71859986,2cyclictest762-21ntpd20:59:301
71859986,2cyclictest762-21ntpd20:51:011
71859986,2cyclictest762-21ntpd20:49:501
71859986,2cyclictest762-21ntpd20:28:361
71819988,0cyclictest784-21snmpd23:44:200
71819988,0cyclictest784-21snmpd21:31:310
71819988,0cyclictest784-21snmpd00:27:480
71819988,0cyclictest784-21snmpd00:16:000
71819988,0cyclictest782-21lldpd23:27:360
71819988,0cyclictest782-21lldpd21:45:530
71819987,1cyclictest784-21snmpd22:53:520
71819987,1cyclictest784-21snmpd22:53:510
71819987,1cyclictest784-21snmpd22:40:290
71819987,1cyclictest784-21snmpd20:58:450
71819987,1cyclictest784-21snmpd19:18:340
71819987,1cyclictest784-21snmpd00:14:230
71819986,2cyclictest784-21snmpd23:01:550
71819986,2cyclictest784-21snmpd22:23:470
71819986,2cyclictest784-21snmpd19:58:010
71819986,2cyclictest784-21snmpd19:42:180
71949970,4cyclictest8064-21ntp_192.168.11419:10:153
71899977,0cyclictest784-21snmpd21:50:002
71899977,0cyclictest784-21snmpd00:33:282
71899977,0cyclictest782-21lldpd23:52:582
71899977,0cyclictest782-21lldpd21:47:492
71899977,0cyclictest782-21lldpd19:34:422
71899977,0cyclictest526-21avahi-daemon19:26:072
71899976,1cyclictest526-21avahi-daemon23:03:172
71899976,1cyclictest4193-21cpuspeed_turbos20:20:112
71899975,2cyclictest784-21snmpd22:49:332
71899975,2cyclictest784-21snmpd20:28:372
71899975,2cyclictest784-21snmpd19:56:522
71899973,3cyclictest959-21ssh00:24:042
71899973,3cyclictest6791-21ssh23:27:082
71899973,3cyclictest482-21ssh21:17:522
71899973,3cyclictest19224-21ssh22:05:302
71859977,0cyclictest782-21lldpd22:40:321
71859977,0cyclictest782-21lldpd21:45:131
71859977,0cyclictest526-21avahi-daemon21:41:291
71859976,1cyclictest784-21snmpd19:16:291
71859975,2cyclictest784-21snmpd20:13:151
71859975,2cyclictest18115-21cat22:05:141
71819977,0cyclictest784-21snmpd22:05:290
71819977,0cyclictest784-21snmpd19:49:230
71819977,0cyclictest784-21snmpd00:03:270
71819977,0cyclictest782-21lldpd23:23:040
71819976,1cyclictest784-21snmpd23:48:370
71819976,1cyclictest784-21snmpd21:26:540
71819976,1cyclictest784-21snmpd20:24:440
71819976,1cyclictest7694-21ntp_kernel_pll_21:55:160
71819976,1cyclictest526-21avahi-daemon23:34:360
71819975,2cyclictest784-21snmpd23:38:470
71819975,2cyclictest784-21snmpd21:24:300
71819975,2cyclictest784-21snmpd20:02:190
71819975,1cyclictest22402-21tr21:05:110
71819973,3cyclictest30098-21ntpq00:20:150
71899966,0cyclictest782-21lldpd22:11:322
71899965,1cyclictest2728-21needreboot00:25:152
71899965,1cyclictest2587-21cpuspeed_turbos21:20:132
71899964,2cyclictest784-21snmpd23:14:002
71899964,2cyclictest784-21snmpd22:55:252
71899964,2cyclictest784-21snmpd20:34:072
71899964,2cyclictest547-21NetworkManager00:36:322
71899963,2cyclictest26138-21ssh21:41:432
71899962,3cyclictest26708-21ssh00:17:042
71899962,3cyclictest10946-21ssh21:27:542
71899962,3cyclictest10879-21ssh00:01:412
71859966,0cyclictest784-21snmpd21:35:091
71859965,1cyclictest784-21snmpd20:07:391
71859963,2cyclictest5196-21ssh21:22:061
71859963,2cyclictest245-20systemd-journal22:39:481
71859963,2cyclictest24106-21ssh23:43:581
71859962,4cyclictest547-21NetworkManager21:55:071
71819966,0cyclictest784-21snmpd21:35:540
71819966,0cyclictest784-21snmpd00:06:200
71819966,0cyclictest782-21lldpd21:00:310
71819966,0cyclictest782-21lldpd00:33:080
71819965,1cyclictest784-21snmpd22:10:470
71819965,1cyclictest784-21snmpd21:41:030
71899955,0cyclictest24064-21idleruntime-cro21:10:012
71899954,1cyclictest784-21snmpd19:46:502
71899954,1cyclictest421ktimers/200:10:142
71899952,3cyclictest14511-21ssh22:01:082
71899952,2cyclictest574-21rs:main2
71899952,2cyclictest30138-21ssh23:19:032
71899951,3cyclictest20674-21ssh21:36:592
71859954,1cyclictest29254-21ntpq20:00:161
71859952,3cyclictest547-21NetworkManager20:40:001
71859952,2cyclictest27584-21sed19:55:211
71859951,3cyclictest6154-21cut22:25:001
71859951,2cyclictest15051-21kworker/u16:2+xprtiod22:46:311
71859950,2cyclictest7715-21grep22:25:201
71819955,0cyclictest784-21snmpd19:10:310
71819953,2cyclictest784-21snmpd20:25:190
71819953,2cyclictest784-21snmpd00:36:490
71819953,2cyclictest547-21NetworkManager23:07:490
71819952,3cyclictest8601-21cat20:30:130
71819952,3cyclictest547-21NetworkManager20:46:360
71819952,3cyclictest547-21NetworkManager20:12:410
71819952,3cyclictest30366-21ssh23:50:070
71819952,3cyclictest29087-21sed22:15:130
71819952,2cyclictest7591-21sh22:25:190
71819952,2cyclictest29906-21grep21:15:140
71819952,2cyclictest27253-21find22:45:120
71819952,2cyclictest26551-21ssh21:11:140
71819952,2cyclictest16009-21ssh22:03:140
71819952,2cyclictest12262-21grep22:30:150
71819950,5cyclictest4092-21ntpq23:55:170
71899942,2cyclictest30876-21cat20:05:182
71899942,2cyclictest28856-21cat20:00:132
71859941,2cyclictest12043-21tune2fs19:20:141
71819942,2cyclictest17893-21date19:35:000
71819942,0cyclictest0-21swapper/020:05:270
71819940,4cyclictest25112-21ntpq19:50:160
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional