You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-04 - 12:40

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3s.osadl.org (updated Sun Jan 04, 2026 00:43:55)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
192839212660,61sleep30-21swapper/319:07:153
19287029461,10sleep10-21swapper/119:07:381
19290527853,19sleep00-21swapper/019:08:050
19284827762,9sleep20-21swapper/219:07:232
193175992320,1cyclictest0-21swapper/323:05:003
193175992118,1cyclictest0-21swapper/320:45:173
193175992017,1cyclictest0-21swapper/322:35:163
193175992017,1cyclictest0-21swapper/320:50:153
19317599201,15cyclictest363444-21sleep21:49:593
193175992010,5cyclictest15850irq/127-ahci[0000:00:17.0]19:14:333
19317599197,9cyclictest399674-21chrt22:20:303
193175991916,1cyclictest0-21swapper/322:55:163
193175991916,1cyclictest0-21swapper/321:30:183
193175991916,1cyclictest0-21swapper/320:05:193
193175991916,1cyclictest0-21swapper/319:30:183
19317599191,14cyclictest0-21swapper/321:55:093
19317599191,13cyclictest0-21swapper/323:50:123
19317599191,13cyclictest0-21swapper/320:35:133
19317599182,10cyclictest518927-21taskset00:15:163
193175991815,1cyclictest0-21swapper/300:00:273
193175991814,2cyclictest0-21swapper/323:29:313
19317599181,14cyclictest0-21swapper/323:40:213
19317599181,13cyclictest447419-21turbostat23:10:003
19317599181,13cyclictest0-21swapper/320:40:143
19317599181,13cyclictest0-21swapper/320:10:123
19317599181,10cyclictest382901-21taskset22:05:183
19317599181,10cyclictest382901-21taskset22:05:183
19317599180,14cyclictest0-21swapper/319:35:123
4886092171,11sleep2488859-21grep23:45:262
2438422171,11sleep2245578-21cut19:55:162
19317599177,6cyclictest421205-21taskset22:45:103
19317599171,4cyclictest0-21swapper/321:25:123
19317599171,4cyclictest0-21swapper/300:10:103
193175991714,1cyclictest0-21swapper/322:25:173
193175991714,1cyclictest0-21swapper/321:15:163
193175991714,1cyclictest0-21swapper/320:30:153
193175991714,1cyclictest0-21swapper/319:20:003
193175991713,2cyclictest0-21swapper/323:15:193
193175991713,2cyclictest0-21swapper/300:30:003
19317599171,13cyclictest0-21swapper/300:30:213
19317599171,12cyclictest499805-21turbostat00:00:003
19317599171,12cyclictest316338-21chrt21:05:103
19317599171,12cyclictest0-21swapper/320:20:123
19317599170,13cyclictest0-21swapper/320:25:203
19317599170,13cyclictest0-21swapper/319:59:423
19317599170,13cyclictest0-21swapper/300:20:173
19317599170,10cyclictest537402-21chrt00:35:003
19317599170,10cyclictest426436-21taskset22:50:083
2939312161,10sleep0293985-21ntpq20:40:240
2005822161,5sleep2200601-21apt-get19:15:122
19317599165,8cyclictest226533-21turbostat19:44:553
19317599161,9cyclictest326716-21taskset21:15:003
193175991611,3cyclictest0-21swapper/300:05:113
19317599160,9cyclictest458137-21taskset23:20:013
19317599160,1cyclictest0-21swapper/323:10:193
19317599160,14cyclictest0-21swapper/321:50:193
19317599160,13cyclictest0-21swapper/323:30:183
19317599160,13cyclictest0-21swapper/322:15:203
19317599160,13cyclictest0-21swapper/322:00:143
19317599160,13cyclictest0-21swapper/321:40:133
19317599160,13cyclictest0-21swapper/321:00:223
19317599160,13cyclictest0-21swapper/320:57:323
19317599160,13cyclictest0-21swapper/320:00:143
19317599160,13cyclictest0-21swapper/319:25:153
19317599151,9cyclictest410123-21taskset22:30:323
19317599151,2cyclictest0-21swapper/323:45:313
193175991512,1cyclictest0-21swapper/321:35:193
193175991512,1cyclictest0-21swapper/321:20:163
19317599151,12cyclictest0-21swapper/319:50:223
19317599150,13cyclictest0-21swapper/323:35:133
5210932142,3sleep1521112-21apt-get00:20:131
5103742141,2sleep10-21swapper/100:09:541
4147352141,9sleep2414787-21ntpq22:35:202
3947732148,3sleep10-21swapper/122:20:001
3888512141,5sleep1389086-21unixbench_multi22:10:271
2744732144,6sleep2274998-21grep20:25:152
193175991411,2cyclictest0-21swapper/319:20:143
19317599140,11cyclictest0-21swapper/322:40:153
19317599140,11cyclictest0-21swapper/320:15:103
5414112130,1sleep20-21swapper/200:35:212
3937902131,7sleep2393837-21ntpq22:15:202
2154702131,7sleep2215676-21users19:25:262
19317599131,8cyclictest232054-21chrt19:45:063
19317599130,11cyclictest0-21swapper/322:10:163
5107092121,7sleep2510927-21sh00:10:122
4283012121,7sleep0430100-21tr22:50:170
4254852121,7sleep0425492-21sensors_temp22:45:250
3634122120,8sleep2363413-21cron21:49:592
3142292121,6sleep2314729-21strings21:00:182
2640142121,7sleep1264815-21kthreadcore20:15:161
2415422121,7sleep0241588-21ls19:50:250
5249322111,5sleep0525073-21ntpq00:20:200
4792772110,1sleep10-21swapper/123:40:121
3781202111,4sleep2378164-21ntpq22:00:212
3780702111,2sleep1171rcu_preempt22:00:201
3639622111,5sleep0364152-21grep21:50:130
3490462111,6sleep1350741-21kthreadcore21:35:191
3429092110,1sleep20-21swapper/221:30:162
3415922111,5sleep2341809-21users21:25:262
3337202111,7sleep1335503-21tr21:20:171
3061922110,7sleep2307345-21grep20:55:162
2220452111,5sleep1222116-21irqstats19:35:151
5362462101,6sleep1536308-21pmu-power00:30:241
5251272101,4sleep2525176-21ntpq00:20:232
5103822104,3sleep2510386-21cron00:09:592
5041702101,4sleep1504219-21ntpq00:00:211
4882852101,5sleep0488335-21ntpq23:45:200
4741182101,4sleep2474327-21hddtemp_smartct23:35:132
4673982101,3sleep2467443-21ntpq23:25:232
4568772101,4sleep0456923-21ntpq23:15:220
4409562101,3sleep2440997-21ntpq23:00:212
4320212101,6sleep0431696-21/usr/sbin/munin22:55:150
4199402102,3sleep1419986-21ntpq22:40:201
4174882102,4sleep2419234-21tr22:40:182
4062592101,5sleep1407918-21cut22:30:181
4045712101,5sleep2404778-21grep22:25:262
3255652101,5sleep0325616-21gcc21:10:220
2693182101,6sleep10-21swapper/120:20:161
2424952101,4sleep1242513-21apt-get19:55:121
2320612101,3sleep00-21swapper/019:45:130
2273142101,5sleep0227409-21cut19:40:190
2112642101,4sleep1211387-21diskstats19:25:131
2098722101,4sleep2209926-21ntpq19:20:222
1983522101,4sleep2198401-21munin-run19:15:012
533297291,4sleep0535005-21tr00:30:180
463404291,4sleep1463609-21expr23:25:141
442430291,4sleep1441935-21apt-get23:05:021
437256291,4sleep0437482-21basename23:00:150
436955291,2sleep10-21swapper/123:00:121
431695291,2sleep10-21swapper/122:55:121
431435291,4sleep1431459-21munin-run22:54:591
399044291,4sleep2399090-21ntpq22:20:222
381120291,4sleep2382900-21tr22:05:192
381120291,4sleep2382900-21tr22:05:182
362397294,2sleep1321rcuc/121:45:211
347959291,2sleep00-21swapper/021:35:140
344155291,4sleep0345935-21cut21:30:170
341243291,4sleep1341296-21ntp_states21:25:211
337828291,4sleep0339273-21tr21:25:160
285126291,4sleep1285197-21irqstats20:35:151
283432291,4sleep0283685-21grep20:30:250
273664291,4sleep0273713-21munin-run20:25:000
257589290,1sleep10-21swapper/120:05:301
253538291,4sleep2254812-21tr20:05:192
248905291,4sleep1250606-21grep20:00:161
216311291,3sleep2216330-21apt-get19:30:122
200908291,4sleep1201122-21if_enp1s019:15:151
531329280,1sleep10-21swapper/100:26:101
527772281,3sleep0526623-21/usr/sbin/munin00:25:160
520457280,4sleep20-21swapper/200:15:372
514591281,3sleep1514640-21ntpq00:10:191
494790281,2sleep2494809-21apt-get23:55:122
494528280,3sleep1494576-21munin-run23:55:001
484108281,3sleep0484189-21latency_hist23:45:010
478828281,3sleep1478973-21date23:40:001
463382281,2sleep00-21swapper/023:25:120
462430281,3sleep2458150-21/usr/sbin/munin23:20:252
456925281,3sleep1456978-21ntp_states23:15:241
447666280,3sleep2447685-21apt-get23:10:112
446460281,3sleep0446517-21ntp_states23:05:210
420945281,3sleep01-21systemd22:45:000
414553281,3sleep0414693-21ntp_192.168.11422:35:190
411284281,3sleep1411732-21grep22:35:151
388379280,3sleep2388520-21ntpq22:10:202
384310280,1sleep00-21swapper/022:09:500
384310280,1sleep00-21swapper/022:09:490
378230280,1sleep00-21swapper/022:00:230
357185281,3sleep2357239-21ld21:40:222
342734281,3sleep1342908-21head21:30:151
336995281,3sleep0221rcuc/021:25:000
331738281,3sleep1331887-21sed21:20:001
331425280,1sleep00-21swapper/021:16:060
327353281,3sleep2328454-21cut21:15:162
316316280,2sleep20-21swapper/221:05:102
300386281,2sleep00-21swapper/020:50:100
284604281,2sleep00-21swapper/020:35:020
284018280,3sleep10-21swapper/120:34:071
279728281,3sleep2280771-21cat20:30:152
262208280,2sleep00-21swapper/020:10:210
248302281,3sleep0248903-21cat20:00:150
226161280,1sleep00-21swapper/019:35:300
215696281,3sleep0215697-21pool-udisksd19:25:320
203276281,3sleep0200583-21/usr/sbin/munin19:15:180
197668281,3sleep1197921-21unixbench_multi19:10:271
537415270,2sleep1537434-21apt-get00:35:111
536950270,5sleep20-21swapper/200:32:292
515854270,2sleep10-21swapper/100:15:111
504904270,1sleep20-21swapper/200:01:392
499457271,3sleep0800-21snmpd23:55:510
490825271,2sleep00-21swapper/023:50:180
483967273,2sleep2664-21gmain23:43:142
473407270,1sleep10-21swapper/123:32:591
469952271,3sleep0471693-21kthreadcore23:30:180
458711271,2sleep1459513-21kthreadcore23:20:161
457519270,1sleep20-21swapper/223:15:412
426166270,1sleep20-21swapper/222:49:512
421487270,3sleep1421610-21tune2fs22:45:141
404902270,4sleep00-21swapper/022:25:280
364243271,2sleep20-21swapper/221:50:152
356999270,1sleep00-21swapper/021:40:190
352232271,2sleep20-21swapper/221:35:282
315782271,2sleep0316280-21gpgv21:05:000
312450271,2sleep10-21swapper/121:00:171
310565270,1sleep00-21swapper/020:59:550
299405270,1sleep00-21swapper/020:45:240
294206271,3sleep1294402-21unixbench-2d20:40:281
268098270,1sleep00-21swapper/020:15:510
263165270,1sleep20-21swapper/220:14:402
258502270,4sleep1258519-21/usr/sbin/munin20:10:141
237273270,1sleep20-21swapper/219:50:032
233112271,2sleep1234822-21kthreadcore19:45:171
221272270,1sleep00-21swapper/019:34:500
205889270,2sleep10-21swapper/119:20:131
542118260,1sleep00-21swapper/000:36:280
531554260,4sleep20-21swapper/200:28:592
506971261,2sleep00-21swapper/000:05:160
500105260,1sleep00-21swapper/000:00:120
494434260,1sleep20-21swapper/223:53:482
485583261,2sleep10-21swapper/123:45:171
473850261,2sleep00-21swapper/023:35:100
473292261,1sleep20-21swapper/223:31:322
463037260,4sleep00-21swapper/023:23:560
452532260,1sleep10-21swapper/123:13:471
452396260,1sleep00-21swapper/023:12:060
432618261,2sleep20-21swapper/222:55:172
430391260,1sleep20-21swapper/222:50:192
410354260,1sleep20-21swapper/222:33:292
405528260,2sleep00-21swapper/022:30:130
402134261,1sleep10-21swapper/122:25:171
395057260,2sleep10-21swapper/122:20:131
394595260,1sleep00-21swapper/022:17:470
379946261,2sleep10-21swapper/122:05:181
379946261,2sleep10-21swapper/122:05:171
363091260,1sleep00-21swapper/021:45:570
353206260,1sleep10-21swapper/121:40:121
335504260,2sleep2335942-21strings21:20:182
326319260,1sleep20-21swapper/221:12:292
326268260,1sleep10-21swapper/121:11:481
321237260,1sleep10-21swapper/121:09:261
315459260,1sleep00-21swapper/021:00:560
310323260,1sleep10-21swapper/120:56:521
302054261,2sleep20-21swapper/220:50:152
300957261,2sleep10-21swapper/120:50:151
285198261,2sleep20-21swapper/220:35:172
278563260,1sleep10-21swapper/120:25:461
273916260,4sleep00-21swapper/020:25:040
273333260,1sleep20-21swapper/220:20:492
268115260,1sleep20-21swapper/220:16:052
252010260,3sleep20-21swapper/220:00:252
241884260,1sleep10-21swapper/119:50:341
231669260,4sleep20-21swapper/219:43:232
222117261,2sleep20-21swapper/219:35:162
221081260,1sleep10-21swapper/119:32:281
205878260,1sleep00-21swapper/019:20:040
194658261,2sleep00-21swapper/019:10:180
499531250,1sleep10-21swapper/123:56:481
446357250,1sleep20-21swapper/223:05:192
395351250,1sleep00-21swapper/022:20:150
385723250,2sleep00-21swapper/022:10:180
373849250,1sleep10-21swapper/121:59:371
369988250,2sleep00-21swapper/021:55:150
369253250,2sleep20-21swapper/221:55:122
364157250,1sleep10-21swapper/121:50:141
299979250,1sleep10-21swapper/120:48:061
293740250,2sleep20-21swapper/220:40:212
253256250,1sleep00-21swapper/020:05:170
247446250,1sleep00-21swapper/019:59:440
227411250,2sleep10-21swapper/119:40:201
515852240,1sleep00-21swapper/000:15:100
510641240,1sleep00-21swapper/000:10:100
295453240,1sleep20-21swapper/220:45:142
232339240,1sleep20-21swapper/219:45:142
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional