You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-11 - 16:37

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Congatec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot8s.osadl.org (updated Wed Mar 11, 2026 12:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
5614349912250,1221cyclictest5971892sleep307:35:003
5614349912060,1202cyclictest7098992sleep308:59:563
7539942230,2sleep20-21swapper/209:29:592
56143499195,10cyclictest967409-21taskset11:50:003
173198190,13rtkit-daemon767467-21/usr/sbin/munin09:35:220
7367942180,1sleep10-21swapper/109:15:251
56143499182,12cyclictest875622-21taskset10:50:013
56143499181,13cyclictest896977-21taskset11:00:223
56143499180,14cyclictest616957-21turbostat07:50:013
9274412171,5sleep0927537-21ls11:20:240
9213682171,7sleep0723136-21diskmemload11:19:360
56143499175,9cyclictest716386-21taskset09:03:123
56143499171,13cyclictest838011-21taskset10:25:123
56143499171,13cyclictest0-21swapper/312:35:113
56143499171,13cyclictest0-21swapper/309:40:223
56143499171,13cyclictest0-21swapper/309:25:123
56143499171,12cyclictest0-21swapper/311:25:003
56143499170,15cyclictest974231-21taskset11:52:003
7539092161,2sleep10-21swapper/109:30:001
56143499164,9cyclictest860437-21turbostat10:40:013
56143499164,9cyclictest696576-21taskset08:48:583
56143499162,11cyclictest696899-21chrt08:50:023
561434991613,1cyclictest0-21swapper/312:32:113
56143499161,12cyclictest807188-21sleep10:05:003
56143499161,12cyclictest0-21swapper/312:10:113
56143499161,12cyclictest0-21swapper/310:40:113
56143499161,12cyclictest0-21swapper/310:30:213
56143499161,12cyclictest0-21swapper/310:15:113
56143499161,12cyclictest0-21swapper/309:55:113
56143499161,12cyclictest0-21swapper/309:20:113
56143499161,12cyclictest0-21swapper/308:30:113
56143499161,12cyclictest0-21swapper/307:55:213
56143499161,12cyclictest0-21swapper/307:10:123
56143499160,12cyclictest570252-21taskset07:15:023
56143499151,3cyclictest0-21swapper/309:30:213
56143499151,3cyclictest0-21swapper/309:15:133
561434991513,0cyclictest0-21swapper/308:06:213
56143499151,2cyclictest0-21swapper/311:10:123
56143499151,2cyclictest0-21swapper/309:35:113
56143499151,2cyclictest0-21swapper/309:10:113
561434991512,1cyclictest0-21swapper/312:21:083
56143499151,12cyclictest0-21swapper/312:00:103
56143499151,12cyclictest0-21swapper/309:50:123
56143499151,11cyclictest0-21swapper/312:25:113
56143499151,11cyclictest0-21swapper/312:15:113
56143499151,11cyclictest0-21swapper/312:05:113
56143499151,11cyclictest0-21swapper/311:40:123
56143499151,11cyclictest0-21swapper/311:35:113
56143499151,11cyclictest0-21swapper/311:30:183
56143499151,11cyclictest0-21swapper/311:15:143
56143499151,11cyclictest0-21swapper/311:05:143
56143499151,11cyclictest0-21swapper/310:55:123
56143499151,11cyclictest0-21swapper/310:20:093
56143499151,11cyclictest0-21swapper/310:10:213
56143499151,11cyclictest0-21swapper/310:05:113
56143499151,11cyclictest0-21swapper/309:45:133
56143499151,11cyclictest0-21swapper/308:25:193
56143499151,11cyclictest0-21swapper/308:10:123
56143499151,11cyclictest0-21swapper/307:50:133
56143499151,11cyclictest0-21swapper/307:25:143
56143499150,12cyclictest603854-21taskset07:40:023
173198150,10rtkit-daemon827901-21latency10:15:170
6789012141,11sleep00-21swapper/008:35:180
56143499141,9cyclictest0-21swapper/311:25:113
56143499141,2cyclictest0-21swapper/311:55:093
56143499141,2cyclictest0-21swapper/308:15:113
56143499141,2cyclictest0-21swapper/308:00:123
56143499141,2cyclictest0-21swapper/307:20:123
56143499141,11cyclictest0-21swapper/310:50:103
561434991411,1cyclictest0-21swapper/308:36:063
56143499141,11cyclictest0-21swapper/308:20:123
56143499141,10cyclictest0-21swapper/308:40:123
56143499141,10cyclictest0-21swapper/307:30:113
9524632131,8sleep0952502-21ssh11:40:120
7462652131,10sleep0746269-21ssh09:24:540
56143499131,10cyclictest0-21swapper/309:05:143
173198130,9rtkit-daemon782490-21awk09:45:210
9901632120,3sleep20-21swapper/212:04:352
8993232121,8sleep0899432-21/usr/sbin/munin11:05:170
56143499121,2cyclictest0-21swapper/307:10:013
173198120,3rtkit-daemon689665-21users08:40:252
173198120,3rtkit-daemon568276-21munin-node11:03:191
173198120,2rtkit-daemon0-21swapper/110:58:141
9448942110,2sleep10-21swapper/111:35:121
8260672110,8sleep1827135-21ssh10:15:171
7846602111,7sleep1784794-21grep09:50:121
6411172110,1sleep20-21swapper/208:05:182
173198110,4rtkit-daemon851914-21kworker/u16:0+rpciod11:27:222
9372532101,5sleep0937285-21apt-get11:30:100
8850802101,6sleep2887031-21grep10:55:182
8433182101,5sleep2843369-21ntpq10:25:202
8356952101,6sleep2835749-21ntpq10:20:182
7749112101,6sleep1774968-21ssh09:40:211
6238152101,5sleep2623847-21apt-get07:55:132
5849422101,6sleep1586075-21kthreadcore07:25:191
5828442100,3sleep1582895-21ntpq07:20:221
173198100,6rtkit-daemon737587-21ssh09:16:582
173198100,5rtkit-daemon996066-21ntpq12:05:190
173198100,5rtkit-daemon805369-21ntpq10:00:200
173198100,3rtkit-daemon921436-21ssh11:19:502
173198100,3rtkit-daemon647-21lldpd11:49:180
173198100,3rtkit-daemon500-21gmain07:26:212
173198100,3rtkit-daemon453-21wpa_supplicant12:14:230
973205291,4sleep0973258-21ntpq11:50:220
958800290,6sleep2723136-21diskmemload11:41:142
935946290,6sleep1935948-21ssh11:26:111
928062291,5sleep1723136-21diskmemload11:21:181
914345290,6sleep1914465-21/usr/sbin/munin11:15:161
913073290,3sleep20-21swapper/211:12:052
882959290,3sleep00-21swapper/010:54:100
866337290,7sleep2860721-21/usr/sbin/munin10:40:232
807464290,5sleep2807490-21ssh10:05:122
775464290,6sleep2775515-21ssh09:40:262
731369291,5sleep0731772-21irqcore09:15:160
723612290,5sleep2723683-21tune2fs09:10:142
703845291,5sleep2704078-21sed08:55:152
670577290,6sleep2671156-21ps08:30:142
662574293,1sleep0171rcu_preempt08:20:220
662479290,5sleep2662531-21ntp_states08:20:212
636106291,5sleep1636239-21sed08:00:251
630438291,4sleep0630471-21apt-get08:00:120
17319890,6rtkit-daemon435-21avahi-daemon10:41:440
17319890,5rtkit-daemon795500-21ssh09:55:150
17319890,5rtkit-daemon680757-21grep08:35:182
997460280,3sleep10-21swapper/112:08:271
973260280,6sleep1973320-21ntp_states11:50:231
967177280,5sleep2967181-21ssh11:49:122
898971281,4sleep1898490-21apt-get11:05:021
891148281,3sleep2891150-21apt-get11:00:112
877319280,6sleep1879095-21tr10:50:151
845853281,5sleep0846848-21/usr/sbin/munin10:30:150
838092280,5sleep0838212-21/usr/sbin/munin10:25:140
799844281,4sleep2799883-21ssh10:00:112
799342281,4sleep1723136-21diskmemload09:59:131
783948280,3sleep20-21swapper/209:48:372
759678280,5sleep1759730-21ntpq09:30:201
756566280,5sleep2758543-21tr09:30:172
753707280,3sleep00-21swapper/009:29:110
724817280,1sleep10-21swapper/109:10:171
656721280,6sleep2656882-21latency_hist08:20:012
623473280,3sleep00-21swapper/007:53:550
567775280,1sleep00-21swapper/007:12:190
561621281,4sleep2561684-21cpuspeed_turbos07:10:132
561585281,3sleep1561618-21apt-get07:10:111
554945280,3sleep20-21swapper/207:05:122
17319880,5rtkit-daemon568276-21munin-node12:19:140
17319880,5rtkit-daemon500-21gmain11:30:122
17319880,4rtkit-daemon649-21ntpd11:20:222
17319880,4rtkit-daemon568276-21munin-node12:22:520
17319880,4rtkit-daemon568276-21munin-node11:59:480
17319880,4rtkit-daemon568276-21munin-node11:35:500
17319880,4rtkit-daemon453-21wpa_supplicant12:34:131
17319880,4rtkit-daemon453-21wpa_supplicant10:13:460
17319880,4rtkit-daemon453-21wpa_supplicant09:33:450
17319880,4rtkit-daemon453-21wpa_supplicant08:50:232
17319880,4rtkit-daemon1028503-21ssh12:28:361
17319880,3rtkit-daemon1043602-21ssh12:37:521
17319880,1rtkit-daemon0-21swapper/010:39:100
17319880,1rtkit-daemon0-21swapper/009:43:120
1013336281,4sleep11013564-21latency_hist12:20:011
1005114280,3sleep10-21swapper/112:13:441
990569270,3sleep2990608-21ssh12:05:112
966380270,1sleep10-21swapper/111:46:091
883804271,3sleep0884180-21sed10:55:160
868287271,3sleep2868322-21apt-get10:45:122
835857271,3sleep1835984-21ssh10:20:221
821181271,3sleep1723136-21diskmemload10:10:321
815088270,1sleep20-21swapper/210:10:132
806113271,3sleep1806117-21ssh10:01:061
792218271,3sleep2792253-21apt-get09:55:102
768757270,1sleep10-21swapper/109:38:461
738992270,4sleep2739007-21apt-get09:20:122
713063271,3sleep00-21swapper/009:00:170
705092270,4sleep0707131-21kthreadcore08:55:170
699456270,4sleep0701624-21cut08:50:160
697113271,3sleep1697271-21/usr/sbin/munin08:50:131
696657270,5sleep1696801-21latency_hist08:50:001
683659270,3sleep1683692-21apt-get08:40:121
676039271,3sleep0676256-21users08:30:250
662701270,4sleep1662825-21cat08:20:241
649249270,4sleep2649353-21awk08:10:232
635883270,4sleep2635936-21ntp_states08:00:222
623558270,4sleep20-21swapper/207:55:002
623444270,3sleep10-21swapper/107:53:311
597202271,3sleep1597235-21apt-get07:35:121
591524270,4sleep1592465-21kthreadcore07:30:161
576461270,4sleep1576700-21tail07:15:251
17319870,4rtkit-daemon647-21lldpd09:01:582
17319870,4rtkit-daemon643-21snmpd11:12:540
17319870,4rtkit-daemon500-21gmain10:47:050
17319870,4rtkit-daemon500-21gmain07:15:252
17319870,4rtkit-daemon435-21avahi-daemon10:20:080
17319870,3rtkit-daemon912735-21ssh11:10:471
17319870,3rtkit-daemon813615-21runrttasks10:05:390
17319870,3rtkit-daemon723136-21diskmemload12:00:050
17319870,3rtkit-daemon694939-21kthreadcore08:45:162
17319870,3rtkit-daemon649-21ntpd09:53:510
17319870,3rtkit-daemon643-21snmpd07:20:092
17319870,3rtkit-daemon609337-21ntpq07:40:212
17319870,3rtkit-daemon453-21wpa_supplicant07:37:082
1044072271,3sleep00-21swapper/012:39:390
1036783271,3sleep2723136-21diskmemload12:35:092
1030151270,4sleep01031379-21cut12:30:140
1013565271,3sleep21013352-21apt-get12:20:022
1011567271,3sleep21011625-21ntpq12:15:212
989381261,2sleep10-21swapper/112:01:371
982461260,4sleep1982465-21ssh11:59:131
953855261,2sleep10-21swapper/111:40:171
938639260,3sleep1940097-21cut11:30:141
874538260,4sleep1874542-21ssh10:46:031
852053260,1sleep10-21swapper/110:32:081
837823261,2sleep10-21swapper/110:25:111
822643260,4sleep2822670-21awk10:15:102
814512261,3sleep1814516-21ssh10:09:081
786915261,2sleep2787742-21kthreadcore09:50:172
718712261,2sleep10-21swapper/109:05:201
639055260,3sleep1641115-21grep08:05:171
615959261,2sleep10-21swapper/107:45:181
17319860,3rtkit-daemon649-21ntpd07:45:012
17319860,3rtkit-daemon568276-21munin-node08:25:322
17319860,2rtkit-daemon783906-21ssh09:48:261
17319860,2rtkit-daemon649-21ntpd09:05:102
17319860,2rtkit-daemon1021114-21ssh12:24:291
17319860,2rtkit-daemon0-21swapper/207:30:172
1036509260,4sleep21036556-21munin-run12:35:012
1028176260,4sleep00-21swapper/012:27:210
1027159260,1sleep20-21swapper/212:25:232
974858250,1sleep20-21swapper/211:54:212
950401251,2sleep20-21swapper/211:35:212
860476250,3sleep1860681-21ssh10:40:021
860380250,3sleep2860384-21ssh10:39:542
859266250,3sleep1859268-21ssh10:35:441
851763250,3sleep20-21swapper/210:31:052
761802250,2sleep2761837-21apt-get09:35:102
746196251,2sleep10-21swapper/109:24:391
717686251,2sleep00-21swapper/009:05:190
716232250,1sleep10-21swapper/109:01:141
709754250,1sleep10-21swapper/108:58:071
690286251,2sleep00-21swapper/008:45:110
650701250,2sleep10-21swapper/108:15:151
645192251,2sleep00-21swapper/008:10:170
637964250,2sleep00-21swapper/008:05:160
629862250,1sleep00-21swapper/007:56:110
603867251,2sleep10-21swapper/107:40:131
583519250,1sleep00-21swapper/007:21:530
576901250,1sleep00-21swapper/007:16:560
1003577250,2sleep21003634-21ssh12:10:192
975300240,2sleep2975304-21ssh11:55:092
929791240,2sleep00-21swapper/011:25:130
900430240,1sleep20-21swapper/211:05:202
896711240,1sleep00-21swapper/011:00:210
881453240,2sleep20-21swapper/210:50:212
728827240,2sleep0728924-21ntpq09:10:200
689916240,3sleep00-21swapper/008:43:310
676354240,1sleep10-21swapper/108:31:341
669110240,1sleep10-21swapper/108:25:211
665057240,2sleep0666868-21kthreadcore08:25:160
650553240,2sleep00-21swapper/008:15:140
630161240,2sleep1630188-21/usr/sbin/munin08:00:011
616593240,3sleep00-21swapper/007:45:410
584021240,1sleep00-21swapper/007:25:130
560386240,1sleep00-21swapper/007:05:220
676996230,1sleep10-21swapper/108:35:111
647199230,1sleep10-21swapper/108:10:181
604172230,1sleep00-21swapper/007:40:160
597431230,1sleep00-21swapper/007:35:150
590620230,1sleep00-21swapper/007:30:110
555250230,1sleep10-21swapper/107:05:151
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional