You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-22 - 00:05

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Congatec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot8s.osadl.org (updated Sat Mar 21, 2026 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4677779920350,2031cyclictest9221902sleep312:24:553
4677779917711,1765cyclictest623009-21sleep09:10:013
467777992220,219cyclictest814806-21turbostat11:19:553
7820702211,8sleep0782111-21/usr/sbin/munin10:50:250
7820702211,8sleep0782111-21/usr/sbin/munin10:50:240
467777992017,1cyclictest0-21swapper/309:50:013
467777992017,1cyclictest0-21swapper/309:35:193
46777799186,9cyclictest467947-21taskset07:10:013
46777799177,7cyclictest753421-21turbostat10:34:593
46777799171,12cyclictest653688-21taskset09:30:003
46777799164,8cyclictest8990272sleep312:09:363
467777991613,1cyclictest0-21swapper/311:38:553
467777991613,1cyclictest0-21swapper/310:20:183
467777991613,1cyclictest0-21swapper/310:17:113
467777991612,1cyclictest0-21swapper/308:05:153
46777799161,12cyclictest508127-21taskset07:39:563
46777799161,11cyclictest913874-21taskset12:18:013
46777799161,11cyclictest854842-21taskset11:40:153
46777799161,11cyclictest713827-21taskset10:06:063
46777799161,11cyclictest697637-21taskset09:55:243
46777799161,11cyclictest597879-21taskset08:50:143
46777799161,11cyclictest576720-21taskset08:30:173
46777799161,11cyclictest476697-21taskset07:15:143
46777799161,10cyclictest7820152sleep310:50:243
46777799161,10cyclictest7820152sleep310:50:233
7761962151,6sleep1776200-21ssh10:49:081
467777991513,1cyclictest0-21swapper/307:41:233
467777991513,0cyclictest0-21swapper/308:43:593
467777991512,1cyclictest0-21swapper/312:30:513
467777991512,1cyclictest0-21swapper/312:29:103
467777991512,1cyclictest0-21swapper/312:13:023
467777991512,1cyclictest0-21swapper/307:55:423
467777991512,1cyclictest0-21swapper/307:30:543
46777799151,11cyclictest943965-21taskset12:36:153
46777799151,11cyclictest883504-21taskset11:59:043
46777799151,11cyclictest829573-21taskset11:23:243
46777799151,11cyclictest805644-21taskset11:05:343
46777799151,11cyclictest769049-21taskset10:45:013
46777799151,11cyclictest745398-21taskset10:28:533
46777799151,11cyclictest659166-21taskset09:30:213
46777799151,11cyclictest630671-21taskset09:15:003
46777799151,11cyclictest539793-21taskset08:01:253
46777799151,11cyclictest518319-21taskset07:45:183
46777799151,10cyclictest874681-21taskset11:50:303
46777799151,10cyclictest807363-21taskset11:10:083
46777799151,10cyclictest794938-21taskset11:00:163
46777799151,10cyclictest762423-21taskset10:40:153
46777799151,10cyclictest759088-21taskset10:35:233
46777799151,10cyclictest6386232sleep309:20:113
46777799151,10cyclictest604096-21chrt08:55:123
46777799151,10cyclictest604096-21chrt08:55:113
46777799151,10cyclictest5913472sleep308:45:143
46777799151,10cyclictest494429-21taskset07:25:203
46777799145,7cyclictest565344-21taskset08:23:003
46777799142,8cyclictest7000852sleep310:00:143
46777799141,9cyclictest482930-21chrt07:20:123
467777991412,0cyclictest0-21swapper/309:50:443
467777991411,1cyclictest0-21swapper/311:47:533
467777991411,1cyclictest0-21swapper/307:50:213
46777799141,10cyclictest890061-21taskset12:00:343
46777799141,10cyclictest836918-21taskset11:27:243
46777799141,10cyclictest790739-21taskset10:57:053
46777799141,10cyclictest721401-21taskset10:10:523
46777799141,10cyclictest674576-21taskset09:40:193
46777799141,10cyclictest631133-21taskset09:15:123
46777799141,10cyclictest616235-21taskset09:00:283
46777799141,10cyclictest584108-21taskset08:35:273
46777799141,10cyclictest571605-21taskset08:26:043
46777799141,10cyclictest558913-21taskset08:17:053
46777799141,10cyclictest552535-21taskset08:11:573
46777799140,11cyclictest838072-21taskset11:30:003
4889682131,8sleep21-21systemd07:25:012
4673972130,3sleep30-21swapper/307:06:463
7154452121,7sleep2715447-21apt-get10:10:122
8995712110,8sleep0899722-21/usr/sbin/munin12:10:130
8835032111,8sleep2883501-21ssh11:59:042
8224322111,3sleep00-21swapper/011:20:000
7514462111,7sleep2746059-21/usr/sbin/munin10:30:242
5005632110,8sleep1500725-21tail07:30:211
4896912110,2sleep2490644-21irqrtprio07:25:162
4616602111,4sleep10-21swapper/107:05:111
9290182101,6sleep0929024-21ssh12:27:190
8920702100,1sleep20-21swapper/212:05:132
8856262100,2sleep2887128-21kthreadcore12:00:182
8300762100,7sleep2830083-21ssh11:24:562
6361012100,3sleep1636142-21ntpq09:15:191
6042562104,3sleep1604449-21gltestperf08:55:151
6042562104,3sleep1604449-21gltestperf08:55:141
5837182101,5sleep2583760-21ntpq08:35:232
5786022101,5sleep1578599-21apt-get08:35:141
5518722101,5sleep0551914-21ntpq08:10:220
5464312101,6sleep2546482-21munin-run08:10:002
4958422100,7sleep0496015-21gltestperf07:30:160
4855332100,7sleep0487531-21tr07:20:180
907782291,6sleep0908707-21/usr/sbin/munin12:15:160
906736290,3sleep1906742-21ssh12:14:371
905918291,5sleep2905924-21ssh12:12:012
873987292,4sleep0874039-21ntpq11:50:210
861780291,5sleep10-21swapper/111:45:161
859522290,5sleep2859529-21ssh11:41:072
853682290,6sleep0622991-21diskmemload11:40:130
843355291,4sleep2843402-21ntp_states11:30:212
836493290,3sleep1836495-21ssh11:26:031
806877290,4sleep20-21swapper/211:09:232
781967290,7sleep20-21swapper/210:50:232
781967290,7sleep20-21swapper/210:50:222
734762291,5sleep0735844-21memory10:20:200
730795292,3sleep1730827-21apt-get10:20:131
723146290,4sleep2723141-21apt-get10:15:122
713891291,6sleep2713897-21ssh10:06:182
692396290,3sleep2692400-21ssh09:55:112
685181291,5sleep0686175-21irqstats09:50:170
677024291,4sleep2677059-21apt-get09:45:132
638666292,3sleep0638660-21apt-get09:20:110
623295291,4sleep2623332-21apt-get09:10:122
604099290,5sleep0604255-21diskstats08:55:130
604099290,5sleep0604255-21diskstats08:55:130
604060291,4sleep2604093-21apt-get08:55:122
604060291,4sleep2604093-21apt-get08:55:112
598375290,6sleep1599215-21cut08:50:181
551731291,5sleep2551825-21ntpq08:10:212
521563291,5sleep0521777-21sed07:50:180
476509290,2sleep00-21swapper/007:15:120
461696291,3sleep20-21swapper/207:05:122
944615280,1sleep0171rcu_preempt12:38:130
930372280,5sleep0930474-21/usr/sbin/munin12:30:150
914922281,3sleep2914915-21apt-get12:20:142
898374281,4sleep1898380-21ssh12:07:381
889353280,5sleep1889404-21ntpq12:00:211
876705281,4sleep1876902-21expr11:55:141
844065280,2sleep00-21swapper/011:30:510
827818281,4sleep0827962-21ntpq11:20:200
822166280,1sleep20-21swapper/211:19:102
813670280,2sleep1813676-21ssh11:11:401
792101281,3sleep1792096-21apt-get11:00:121
775421280,1sleep00-21swapper/010:46:510
761643281,4sleep1761848-21hddtemp_smartct10:40:151
743419282,3sleep0743572-21needreboot10:25:220
731809283,3sleep2732858-21kthreadcore10:20:182
725879280,5sleep0727765-21cut10:15:190
707371280,5sleep1707457-21appstreamcli10:05:011
706609280,5sleep2706613-21ssh10:02:382
705526280,5sleep0705662-21sed10:00:270
697695281,4sleep0697732-21ntpq09:55:250
682251281,4sleep0682298-21ntpq09:45:220
666879280,4sleep1666923-21ntpq09:35:221
646316281,3sleep1646311-21apt-get09:25:141
641359280,5sleep2643269-21cut09:20:182
636881281,4sleep2636887-21ssh09:15:482
610503281,3sleep1610500-21apt-get09:00:111
602869281,4sleep2602909-21ntpq08:50:232
592771280,4sleep0594396-21sed08:45:180
584986280,5sleep0585057-21sort08:40:140
574721281,5sleep20-21swapper/208:30:172
513578280,4sleep0513623-21ntpq07:40:220
507258281,4sleep0507297-21/usr/sbin/munin07:35:220
476546280,4sleep2476696-21df_inode07:15:142
474110281,4sleep1474418-21kthreadcore07:15:011
936392271,3sleep2936396-21ssh12:31:252
929927270,5sleep11-21systemd12:30:011
912413270,4sleep1912459-21ssh12:15:211
890431272,3sleep0622991-21diskmemload12:01:440
867253270,5sleep2867259-21ssh11:46:142
859432270,4sleep1859436-21ssh11:40:461
851044270,3sleep2851081-21ssh11:35:232
850986271,3sleep1851040-21ntpq11:35:221
843310270,4sleep1843353-21ntpq11:30:201
806851270,5sleep1806855-21ssh11:09:181
799778271,2sleep00-21swapper/011:05:120
791042270,1sleep20-21swapper/210:57:572
784769271,3sleep0784913-21/usr/sbin/munin10:55:170
783239270,3sleep1783245-21ssh10:52:261
783239270,3sleep1783245-21ssh10:52:251
760404271,4sleep2760408-21ssh10:37:512
753808271,3sleep1753920-21entropy10:35:141
753694270,1sleep00-21swapper/010:35:080
752566270,5sleep1752572-21ssh10:32:241
728445270,5sleep1728489-21/usr/sbin/munin10:15:241
699470270,4sleep10-21swapper/109:59:151
687545271,3sleep20-21swapper/209:50:182
674624271,4sleep0674677-21ntp_states09:40:210
669382271,2sleep20-21swapper/209:40:112
669340271,3sleep1669379-21apt-get09:40:101
661835270,4sleep2661927-21fschecks_time09:35:142
654160270,4sleep1654364-21hddtemp_smartct09:30:151
644712270,5sleep1644718-21ssh09:21:181
596499270,4sleep1596541-21ntp_states08:45:231
590133270,4sleep1590184-21ntp_states08:40:221
571119270,4sleep0571225-21/usr/sbin/munin08:25:260
565173274,2sleep0474068-21kworker/0:0+events_freezable08:20:480
559461270,4sleep1559617-21diskstats08:20:131
516436271,3sleep2518317-21kthreadcore07:45:182
513440271,3sleep2513527-21ntpq07:40:202
508418271,3sleep1508451-21apt-get07:40:111
482893271,3sleep1482926-21apt-get07:20:111
473125270,4sleep2473175-21ntpq07:10:212
467960271,3sleep1467993-21apt-get07:10:131
464712270,4sleep0466500-21sed07:05:170
943960260,3sleep2943964-21ssh12:36:152
940170260,4sleep1942079-21kthreadcore12:35:171
937273260,1sleep10-21swapper/112:34:051
929266260,1sleep20-21swapper/212:28:012
921339260,3sleep1921343-21ssh12:22:221
907402260,3sleep2907561-21gltestperf12:15:132
883068260,1sleep00-21swapper/011:57:500
875011260,4sleep20-21swapper/211:51:262
870290261,3sleep1871744-21cut11:50:181
867800260,1sleep00-21swapper/011:47:510
852360260,1sleep00-21swapper/011:37:420
836463260,3sleep0622991-21diskmemload11:25:580
828609260,1sleep10-21swapper/111:20:301
820437260,3sleep1820487-21ps11:15:241
798816260,1sleep00-21swapper/011:03:120
793550261,2sleep20-21swapper/211:00:162
790439260,3sleep1790445-21ssh10:56:071
775481260,1sleep20-21swapper/210:47:012
766783261,3sleep2766864-21ssh10:40:232
763285260,3sleep0765100-21tr10:40:180
739948260,4sleep2741617-21kthreadcore10:25:202
738426260,1sleep10-21swapper/110:25:071
721319262,2sleep0467773-21cyclictest10:10:370
714944260,1sleep10-21swapper/110:09:341
714261260,1sleep00-21swapper/010:07:280
683071260,1sleep10-21swapper/109:46:041
659225260,3sleep0659270-21ntpq09:30:220
653952261,2sleep20-21swapper/209:30:122
647759260,3sleep2649156-21cut09:25:192
631328260,3sleep0631473-21cut09:15:140
616891260,4sleep1617030-21cpuspeed_turbos09:05:121
610467261,2sleep00-21swapper/009:00:110
597684260,4sleep00-21swapper/008:50:130
597096260,1sleep20-21swapper/208:46:182
584948260,4sleep2584949-21/usr/sbin/munin08:40:112
584300260,1sleep00-21swapper/008:35:280
577254260,3sleep0577348-21ntpq08:30:190
533951261,2sleep0533984-21apt-get08:00:120
533475260,1sleep00-21swapper/007:57:340
532820260,3sleep2532863-21/usr/sbin/munin07:55:252
526237260,3sleep1526325-21ntpq07:50:221
522626261,2sleep20-21swapper/207:50:192
520866260,3sleep0520946-21appstreamcli07:50:020
519899260,3sleep1519944-21ld07:45:211
502032261,2sleep1502029-21apt-get07:35:111
495683261,2sleep20-21swapper/207:30:152
494474260,3sleep0494522-21ntpq07:25:220
489235261,2sleep1489268-21apt-get07:25:111
473791260,1sleep00-21swapper/007:10:560
897006250,2sleep0897053-21ntpq12:05:200
836496250,3sleep20-21swapper/211:26:032
807371251,2sleep00-21swapper/011:10:100
751937250,3sleep00-21swapper/010:30:300
721705250,1sleep10-21swapper/110:11:431
691671250,3sleep10-21swapper/109:53:501
661683250,2sleep00-21swapper/009:35:120
624795250,2sleep0626435-21ps09:10:180
623290250,3sleep10-21swapper/109:10:111
616866250,2sleep00-21swapper/009:05:110
616367250,1sleep20-21swapper/209:02:092
568882251,2sleep20-21swapper/208:25:202
554495250,2sleep10-21swapper/108:15:161
553072250,2sleep00-21swapper/008:15:120
540373250,2sleep00-21swapper/008:05:120
539812250,1sleep20-21swapper/208:01:432
533987250,2sleep10-21swapper/108:00:131
914533240,1sleep00-21swapper/012:20:000
807383240,1sleep20-21swapper/211:10:122
652732240,1sleep00-21swapper/009:27:180
622809240,2sleep2431ktimers/209:07:542
573630240,2sleep1574718-21kthreadcore08:30:161
571907240,2sleep11-21systemd08:30:011
564614241,1sleep20-21swapper/208:20:222
552771240,2sleep1552981-21latency_hist08:15:011
545749240,3sleep10-21swapper/108:05:251
503455240,2sleep20-21swapper/207:35:152
553229230,1sleep20-21swapper/208:15:132
527739230,1sleep10-21swapper/107:55:151
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional