You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-06 - 13:24

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Sun Sep 06, 2026 00:44:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1999699297,21cyclictest34-21ksoftirqd/321:10:003
19995992620,5cyclictest606-21dbus-daemon23:50:002
1999599260,22cyclictest3282-21ssh21:28:422
1999599260,1cyclictest13662-21diskmemload00:33:282
19996992522,2cyclictest30395-21head22:21:443
19995992522,3cyclictest13662-21diskmemload22:39:142
19995992522,3cyclictest0-21swapper/223:39:182
19995992522,2cyclictest20109-21ssh22:41:542
19995992520,4cyclictest3212-21apt-get19:41:342
1999599250,25cyclictest0-21swapper/222:23:462
1999599250,24cyclictest0-21swapper/223:42:292
1999599250,24cyclictest0-21swapper/221:20:502
1999599250,24cyclictest0-21swapper/200:07:112
1999399250,24cyclictest13662-21diskmemload21:58:390
19996992418,0cyclictest0-21swapper/322:05:253
19995992421,2cyclictest0-21swapper/222:26:482
19995992420,4cyclictest13662-21diskmemload23:24:092
19995992419,4cyclictest1372-21ssh22:55:212
1999599240,24cyclictest13662-21diskmemload23:13:492
1999599240,24cyclictest13662-21diskmemload23:13:492
1999599240,23cyclictest20660-21ssh21:16:152
1999599240,23cyclictest0-21swapper/221:53:362
1999599240,20cyclictest0-21swapper/223:56:402
1999599240,1cyclictest27214-21sh22:49:072
1999599240,0cyclictest0-21swapper/223:25:352
1999599240,0cyclictest0-21swapper/223:08:442
1999599240,0cyclictest0-21swapper/222:19:122
1999599240,0cyclictest0-21swapper/221:10:062
19994992423,1cyclictest13662-21diskmemload00:37:391
19994992422,1cyclictest391-21ssh21:55:271
1999499240,23cyclictest29727-21ssh00:20:151
1999499240,23cyclictest13662-21diskmemload21:50:161
1999499240,23cyclictest0-21swapper/122:04:301
19993992422,2cyclictest3606-21ssh22:26:490
19993992422,1cyclictest0-21swapper/021:24:450
1999399240,24cyclictest0-21swapper/021:18:530
1999399240,23cyclictest31529-21sh21:53:510
1999399240,23cyclictest13988-21sh23:06:390
1999399240,23cyclictest13662-21diskmemload23:57:500
1999399240,23cyclictest13662-21diskmemload21:27:100
1999399240,0cyclictest0-21swapper/023:15:480
19996992323,0cyclictest13662-21diskmemload23:19:503
19996992323,0cyclictest13662-21diskmemload23:04:243
19996992323,0cyclictest13662-21diskmemload21:52:423
19996992323,0cyclictest0-21swapper/323:59:163
19996992323,0cyclictest0-21swapper/323:49:323
19996992323,0cyclictest0-21swapper/323:31:093
19996992323,0cyclictest0-21swapper/300:08:513
19996992322,1cyclictest19624-21perf20:15:013
19996992321,1cyclictest4073-21rm21:58:103
19996992321,1cyclictest23669-21apt-get22:46:353
19996992321,1cyclictest0-21swapper/321:21:593
19996992320,2cyclictest30787-21sh23:21:413
19996992320,2cyclictest22908-21ssh00:13:333
1999699230,23cyclictest13662-21diskmemload22:38:023
1999699230,23cyclictest13662-21diskmemload00:26:073
1999699230,23cyclictest0-21swapper/323:51:313
1999699230,22cyclictest7816-21sh21:32:153
1999699230,22cyclictest5854-21rm23:28:333
1999699230,22cyclictest17876-21ssh00:38:553
1999699230,22cyclictest16368-21ssh23:37:553
1999699230,22cyclictest13662-21diskmemload22:27:383
1999699230,1cyclictest28656-21sh22:50:493
1999699230,0cyclictest0-21swapper/322:15:073
1999699230,0cyclictest0-21swapper/321:49:143
19995992323,0cyclictest0-21swapper/221:40:152
19995992323,0cyclictest0-21swapper/200:29:502
19995992321,1cyclictest9478-21ssh22:02:582
19995992321,1cyclictest0-21swapper/200:18:572
19995992320,3cyclictest0-21swapper/223:01:152
19995992320,2cyclictest0-21swapper/200:22:292
19995992319,3cyclictest0-21swapper/223:50:272
19995992319,1cyclictest0-21swapper/200:11:562
19995992319,0cyclictest13662-21diskmemload22:31:312
1999599230,2cyclictest30715-21switchtime22:51:442
1999599230,2cyclictest16555-21ssh00:36:542
1999599230,2cyclictest13404-21ssh21:37:262
1999599230,23cyclictest8088-21ssh23:31:312
1999599230,22cyclictest8291-21ssh00:00:182
1999599230,22cyclictest15716-21ssh22:09:132
1999599230,0cyclictest0-21swapper/223:15:592
1999599230,0cyclictest0-21swapper/221:55:492
1999599230,0cyclictest0-21swapper/221:47:302
19994992321,1cyclictest8033-21sh22:31:371
19994992321,1cyclictest3629-21apt-get23:56:351
19994992321,1cyclictest20125-21ssh22:41:551
19994992321,1cyclictest0-21swapper/123:20:031
19994992321,1cyclictest0-21swapper/119:35:021
19994992321,1cyclictest0-21swapper/100:10:321
19994992320,2cyclictest31415-21ssh21:25:081
1999499230,2cyclictest774-21runrttasks23:26:411
1999499230,2cyclictest27803-21turbostat.cron22:50:001
1999499230,2cyclictest10499-21sh21:35:491
1999499230,23cyclictest0-21swapper/122:10:351
1999499230,22cyclictest978-21ssh22:25:281
1999499230,22cyclictest27584-21latency_hist19:25:011
1999499230,22cyclictest2235-21/usr/sbin/munin22:56:431
1999499230,22cyclictest11917-21ssh22:35:351
1999499230,22cyclictest11284-21ssh00:32:021
1999499230,22cyclictest0-21swapper/119:46:341
1999499230,22cyclictest0-21swapper/100:01:401
1999499230,0cyclictest13662-21diskmemload22:16:021
1999499230,0cyclictest0-21swapper/123:07:131
1999499230,0cyclictest0-21swapper/122:23:111
1999499230,0cyclictest0-21swapper/121:10:171
19993992323,0cyclictest13662-21diskmemload22:41:360
19993992323,0cyclictest0-21swapper/023:37:050
19993992323,0cyclictest0-21swapper/023:27:460
19993992323,0cyclictest0-21swapper/021:32:110
1999399232,21cyclictest62550irq/127-eno122:09:510
19993992322,1cyclictest13662-21diskmemload00:35:260
19993992321,2cyclictest0-21swapper/000:15:420
19993992321,1cyclictest30322-21ntp_states22:51:420
19993992321,1cyclictest13662-21diskmemload22:36:190
19993992321,1cyclictest0-21swapper/022:11:100
19993992321,1cyclictest0-21swapper/021:39:450
19993992320,2cyclictest27296-21latency_hist22:20:000
19993992320,2cyclictest1669-21turbostat.cron20:45:000
1999399231,21cyclictest11931-21sh23:34:290
1999399231,1cyclictest62550irq/127-eno122:47:370
1999399231,1cyclictest12440-21sleep20:00:010
1999399230,2cyclictest0-21swapper/023:44:330
1999399230,23cyclictest13662-21diskmemload22:33:030
1999399230,23cyclictest13662-21diskmemload00:20:010
1999399230,22cyclictest6391-21wc23:00:000
1999399230,22cyclictest32419-21apt-get19:35:010
1999399230,22cyclictest25273-21sh21:47:480
1999399230,22cyclictest0-21swapper/000:31:360
1999399230,0cyclictest13662-21diskmemload21:41:370
19996992222,0cyclictest13662-21diskmemload21:26:593
19996992222,0cyclictest0-21swapper/323:10:153
19996992222,0cyclictest0-21swapper/323:10:153
19996992222,0cyclictest0-21swapper/322:04:583
19996992220,1cyclictest21513-21sh21:16:363
1999699220,22cyclictest0-21swapper/322:31:503
1999699220,21cyclictest29660-21apt-get20:36:343
1999699220,21cyclictest2270-21apt-get22:56:393
1999699220,21cyclictest20989-21sh22:13:563
1999699220,21cyclictest18267-21ssh23:40:123
1999699220,21cyclictest16628-21ssh21:40:573
1999699220,21cyclictest0-21swapper/322:40:293
1999699220,21cyclictest0-21swapper/321:35:563
1999699220,21cyclictest0-21swapper/321:10:113
1999699220,21cyclictest0-21swapper/300:00:413
1999699220,1cyclictest9480-21cstates00:31:363
1999699220,0cyclictest0-21swapper/300:22:513
1999599220,21cyclictest22719-21/usr/sbin/munin19:16:342
1999599220,21cyclictest0-21swapper/221:06:342
1999599220,1cyclictest5440-21ssh21:31:272
19994992222,0cyclictest13662-21diskmemload23:00:271
19994992222,0cyclictest0-21swapper/122:51:111
19994992222,0cyclictest0-21swapper/121:45:161
19994992222,0cyclictest0-21swapper/121:40:051
19994992222,0cyclictest0-21swapper/100:19:361
19994992220,1cyclictest19713-21ssh21:15:061
19994992220,1cyclictest0-21swapper/123:50:401
19994992220,1cyclictest0-21swapper/123:45:501
1999499220,22cyclictest0-21swapper/123:30:091
1999499220,22cyclictest0-21swapper/100:26:341
1999499220,22cyclictest0-21swapper/100:05:281
1999499220,21cyclictest0-21swapper/123:40:151
1999499220,1cyclictest0-21swapper/123:35:561
1999499220,1cyclictest0-21swapper/122:06:031
1999499220,1cyclictest0-21swapper/121:21:181
19993992222,0cyclictest0-21swapper/000:11:370
19993992220,1cyclictest24782-21apt-get19:20:020
19993992220,1cyclictest0-21swapper/021:11:000
1999399220,22cyclictest13662-21diskmemload00:28:320
1999399220,21cyclictest622-21gdbus19:15:010
1999399220,21cyclictest622-21gdbus19:15:010
1999399220,21cyclictest0-21swapper/023:45:030
1999399220,21cyclictest0-21swapper/022:01:450
1999399220,21cyclictest0-21swapper/019:26:350
1999399220,1cyclictest13662-21diskmemload23:51:170
1999399220,1cyclictest0-21swapper/023:00:550
1999399220,1cyclictest0-21swapper/000:07:280
19996992120,1cyclictest22388-21perf19:15:013
19996992120,1cyclictest22388-21perf19:15:013
19996992119,1cyclictest595-21apt-get19:35:013
1999699210,21cyclictest13662-21diskmemload23:05:113
1999699210,21cyclictest0-21swapper/300:15:103
1999699210,20cyclictest0-21swapper/319:56:353
1999599210,20cyclictest32078-21apt-get20:41:352
1999599210,20cyclictest0-21swapper/219:20:022
19994992120,1cyclictest4453-21apt-get20:51:331
19994992120,1cyclictest11659-21apt-get21:06:341
19994992120,1cyclictest0-21swapper/119:31:341
19994992119,1cyclictest0-21swapper/119:45:001
1999499210,21cyclictest18727-21apt-get23:11:361
1999499210,21cyclictest18727-21apt-get23:11:361
1999499210,20cyclictest0-21swapper/120:46:341
1999499210,20cyclictest0-21swapper/120:46:341
19993992119,1cyclictest606-21dbus-daemon20:05:010
1999399210,21cyclictest0-21swapper/023:20:010
19996992020,0cyclictest0-21swapper/320:01:343
19995992020,0cyclictest0-21swapper/220:46:342
19995992020,0cyclictest0-21swapper/220:46:342
19995992020,0cyclictest0-21swapper/219:50:002
19995992020,0cyclictest0-21swapper/219:11:342
19995992020,0cyclictest0-21swapper/219:11:342
1999599200,19cyclictest32419-21apt-get19:35:012
1999599200,0cyclictest0-21swapper/220:30:002
1999599200,0cyclictest0-21swapper/219:31:342
19994992020,0cyclictest0-21swapper/120:26:341
19993992018,1cyclictest26805-21cron20:30:000
1999399200,20cyclictest0-21swapper/019:50:000
19996991919,0cyclictest0-21swapper/320:25:003
19996991919,0cyclictest0-21swapper/319:46:343
19996991918,1cyclictest30410-21apt-get19:31:343
1999699190,19cyclictest0-21swapper/319:20:023
19995991919,0cyclictest0-21swapper/220:51:332
19995991919,0cyclictest0-21swapper/220:11:332
19994991919,0cyclictest0-21swapper/120:41:341
19994991918,1cyclictest20358-21apt-get19:11:341
19994991918,1cyclictest20358-21apt-get19:11:341
19994991917,1cyclictest0-21swapper/121:05:011
19994991917,1cyclictest0-21swapper/120:35:001
19994991916,2cyclictest12427-21dump-pmu-power20:00:001
19993991919,0cyclictest0-21swapper/020:51:330
19993991919,0cyclictest0-21swapper/020:46:340
19993991919,0cyclictest0-21swapper/020:46:340
19993991917,1cyclictest20000-21apt-get20:16:340
19996991818,0cyclictest0-21swapper/320:51:333
19994991817,1cyclictest0-21swapper/119:51:351
1999499180,2cyclictest22016-21systemctl20:20:001
19993991818,0cyclictest0-21swapper/021:06:340
19993991818,0cyclictest0-21swapper/020:31:360
19993991817,1cyclictest1649-21cron20:15:010
19993991817,1cyclictest15202-21apt-get20:06:330
19996991717,0cyclictest0-21swapper/320:31:363
19995991717,0cyclictest27211-21apt-get20:31:362
19995991717,0cyclictest0-21swapper/220:21:352
19995991717,0cyclictest0-21swapper/220:06:332
19994991717,0cyclictest0-21swapper/120:01:341
19994991715,1cyclictest8894-21sendmail-msp21:00:011
19994991714,2cyclictest783-21snmpd19:15:051
19996991616,0cyclictest6853-21apt-get20:56:363
19996991616,0cyclictest0-21swapper/321:01:333
19996991616,0cyclictest0-21swapper/320:26:343
19995991616,0cyclictest0-21swapper/220:56:362
19995991616,0cyclictest0-21swapper/220:36:342
19995991613,2cyclictest7885-21chrt19:50:302
19994991614,1cyclictest17596-21apt-get20:11:331
19994991614,1cyclictest0-21swapper/119:26:361
1999499160,15cyclictest24427-21cron20:25:001
19993991616,0cyclictest0-21swapper/019:41:340
19993991616,0cyclictest0-21swapper/019:31:340
19993991615,1cyclictest9266-21apt-get21:01:330
1999399160,15cyclictest0-21swapper/020:36:380
1999699150,1cyclictest8804-21irqstats19:51:393
1999599156,7cyclictest12811-21apt-get20:01:342
19994991515,0cyclictest0-21swapper/120:36:341
19993991515,0cyclictest0-21swapper/020:56:360
19996991414,0cyclictest0-21swapper/320:15:033
1999699140,14cyclictest0-21swapper/320:06:093
1999699140,14cyclictest0-21swapper/319:41:133
1999699140,14cyclictest0-21swapper/319:25:463
1999699140,0cyclictest0-21swapper/320:45:283
1999699140,0cyclictest0-21swapper/320:45:283
1999699140,0cyclictest0-21swapper/319:15:063
19995991414,0cyclictest0-21swapper/219:56:032
1999599140,14cyclictest0-21swapper/219:25:202
1999599140,0cyclictest0-21swapper/221:00:242
1999599140,0cyclictest0-21swapper/220:15:052
19993991414,0cyclictest0-21swapper/019:50:010
1999399140,14cyclictest0-21swapper/019:16:070
1999399140,0cyclictest0-21swapper/020:20:110
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional