You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-09 - 14:08

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot3s.osadl.org (updated Fri Jan 09, 2026 12:43:56)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
640564212566,55sleep30-21swapper/307:05:463
64056128355,20sleep00-21swapper/007:05:430
64063628267,9sleep20-21swapper/207:06:412
64067927762,9sleep10-21swapper/107:07:121
7917432270,3sleep2791747-21fschecks_count09:30:132
641039992623,1cyclictest0-21swapper/308:30:003
641039992522,1cyclictest0-21swapper/308:45:113
64103999240,20cyclictest733618-21chrt08:34:593
64103999231,18cyclictest0-21swapper/309:40:113
641039992219,1cyclictest0-21swapper/312:15:253
641039992219,1cyclictest0-21swapper/310:00:003
641039992219,1cyclictest0-21swapper/310:00:003
64103999212,12cyclictest926467-21taskset11:35:153
641039992118,1cyclictest0-21swapper/312:00:253
641039992018,1cyclictest0-21swapper/307:25:013
641039992017,1cyclictest0-21swapper/311:55:173
64103999200,14cyclictest8336222sleep310:10:113
641039991916,1cyclictest0-21swapper/308:05:143
641039991916,1cyclictest0-21swapper/307:36:303
641039991915,2cyclictest0-21swapper/308:05:013
64103999191,14cyclictest0-21swapper/312:30:053
64103999182,10cyclictest738915-21taskset08:40:003
641039991815,1cyclictest0-21swapper/311:00:003
641039991815,1cyclictest0-21swapper/309:45:103
641039991815,1cyclictest0-21swapper/309:10:163
641039991815,1cyclictest0-21swapper/308:55:133
641039991814,2cyclictest0-21swapper/311:15:163
64103999181,14cyclictest0-21swapper/311:05:123
64103999181,13cyclictest717901-21sleep08:20:003
64103999181,13cyclictest0-21swapper/311:30:113
64103999181,13cyclictest0-21swapper/309:00:103
9585032170,2sleep20-21swapper/212:05:212
9585032170,2sleep20-21swapper/212:05:212
64103999177,6cyclictest685814-21taskset07:46:413
64103999174,11cyclictest0-21swapper/309:30:203
641039991715,1cyclictest0-21swapper/312:10:163
641039991715,1cyclictest0-21swapper/307:40:153
641039991714,1cyclictest0-21swapper/312:25:173
641039991713,1cyclictest0-21swapper/311:40:223
64103999171,14cyclictest0-21swapper/307:30:103
64103999171,13cyclictest0-21swapper/310:40:113
64103999171,13cyclictest0-21swapper/310:05:193
64103999171,13cyclictest0-21swapper/308:20:253
64103999171,12cyclictest938376-21chrt11:50:103
64103999171,12cyclictest901452-21turbostat11:15:013
64103999171,10cyclictest853090-21taskset10:25:173
64103999170,13cyclictest0-21swapper/311:00:103
64103999170,10cyclictest790956-21taskset09:27:263
7535462161,4sleep2753595-21ntpq08:50:182
641039991614,1cyclictest0-21swapper/307:55:183
64103999161,3cyclictest959443-21sleep12:10:013
64103999161,3cyclictest959443-21sleep12:10:003
641039991613,2cyclictest0-21swapper/310:35:143
641039991613,1cyclictest0-21swapper/310:20:013
641039991612,2cyclictest0-21swapper/311:45:133
64103999161,13cyclictest0-21swapper/312:20:123
64103999161,12cyclictest691337-21sleep07:55:003
64103999161,11cyclictest770217-21turbostat09:10:003
64103999161,11cyclictest663380-21chrt07:25:213
64103999161,11cyclictest0-21swapper/308:53:543
64103999161,10cyclictest828105-21taskset10:04:253
64103999161,10cyclictest812665-21taskset09:50:023
64103999160,13cyclictest0-21swapper/312:35:103
64103999160,13cyclictest0-21swapper/311:20:173
64103999160,13cyclictest0-21swapper/308:40:153
9389002151,9sleep2939408-21grep11:50:142
8428722151,10sleep0842991-21grep10:15:210
8076492151,10sleep2807761-21cut09:45:132
64103999151,9cyclictest874874-21taskset10:45:293
64103999151,9cyclictest847819-21taskset10:20:163
64103999151,9cyclictest785637-21taskset09:21:003
641039991513,1cyclictest0-21swapper/311:25:163
64103999151,2cyclictest0-21swapper/310:33:163
64103999151,11cyclictest6461762sleep307:14:143
64103999151,10cyclictest648331-21chrt07:15:033
641039991510,3cyclictest0-21swapper/310:50:123
64103999150,13cyclictest0-21swapper/308:10:123
64103999150,12cyclictest0-21swapper/309:35:123
64103999150,12cyclictest0-21swapper/309:15:133
9739452141,9sleep1974034-21ntpq12:20:191
8962032141,8sleep0896230-21perf11:10:000
8797982141,8sleep2879900-21sed10:50:262
7553162141,9sleep2755429-21tr08:55:152
7171652141,8sleep0717223-21cat08:15:250
7169642141,3sleep1717055-21ps08:15:221
6649782147,3sleep2665149-21cat07:30:142
9844952122,5sleep0984541-21ntpq12:30:200
7462022121,7sleep10-21swapper/108:45:161
7391352122,5sleep1739154-21apt-get08:40:111
7341632120,7sleep2734372-21expr08:35:112
6746572121,2sleep2674564-31sort07:36:392
6538242121,8sleep10-21swapper/107:20:111
9863192110,7sleep1986904-21grep12:35:151
9541412110,1sleep00-21swapper/012:04:350
8650052112,4sleep1865024-21apt-get10:40:111
7970632111,6sleep1797214-21/usr/sbin/munin09:35:141
7914472111,5sleep0791699-21df09:30:120
6973892111,6sleep1698114-21kthreadcore08:00:151
6759592111,5sleep0676063-21irqstats07:40:140
6746452111,5sleep0674605-31find07:36:330
6504592111,6sleep0652263-21kthreadcore07:15:170
9701852101,4sleep2970395-21df_inode12:20:122
9374082101,6sleep2937630-21unixbench_multi11:45:252
9370792101,4sleep0937125-21ntpq11:45:190
9161732101,4sleep1916219-21ntpq11:25:201
9108572101,5sleep0910909-21collect211:20:180
9004642101,4sleep1900505-21ntpq11:10:211
8964852101,4sleep0896740-21df11:10:120
8760032101,6sleep0876117-21kthreadcore10:50:160
8750542100,1sleep00-21swapper/010:47:480
8058632101,4sleep1805896-21ntpq09:40:191
7814782101,4sleep0781549-21irqstats09:20:140
7794662101,5sleep0641030-21cyclictest09:15:170
7655082100,6sleep2765603-21fschecks_time09:05:132
7076222101,4sleep0707880-21df_inode08:10:120
6890102101,4sleep1690145-21memory07:50:171
986905291,4sleep2988603-21grep12:35:152
968782291,4sleep1968830-21ntpq12:15:181
954458291,4sleep0954713-21df12:05:110
954458291,4sleep0954713-21df12:05:110
948899291,4sleep2948983-21sort12:00:012
941101291,4sleep0942191-21memory11:50:160
927957291,3sleep2928209-21df11:40:122
927935291,3sleep10-21swapper/111:40:111
923257291,4sleep1924792-21cut11:35:141
922654291,3sleep0922673-21apt-get11:35:090
916227291,4sleep2916275-21/usr/sbin/munin11:25:212
908796291,4sleep2910587-21tr11:20:162
901693291,3sleep2901712-21apt-get11:15:112
885992291,3sleep0886011-21apt-get11:00:100
885390290,1sleep00-21swapper/010:55:570
880748291,3sleep2880767-21apt-get10:55:112
859746292,3sleep1859765-21apt-get10:35:101
855444291,4sleep0857125-21tr10:30:170
853525290,1sleep00-21swapper/010:25:230
851885291,3sleep2853089-21memory10:25:172
823480291,4sleep2823202-21/usr/sbin/munin10:00:122
822921291,4sleep2822935-21idleruntime-cro10:00:002
822921291,4sleep2822935-21idleruntime-cro10:00:002
813396291,4sleep0815039-21tr09:50:160
812141291,4sleep0812195-21apt-config09:50:000
795395291,3sleep1795443-21ntpq09:30:211
790151291,4sleep2790203-21ntpq09:25:212
765228291,3sleep00-21swapper/009:05:110
763980291,3sleep1764026-21ntpq09:00:191
760040291,3sleep0760059-21apt-get09:00:100
737903291,4sleep1737952-21ntpq08:35:181
723132291,3sleep2723180-21munin-run08:25:012
702597291,4sleep1702730-21tail08:05:121
701777291,4sleep2702209-21base6408:05:012
700658291,4sleep0700768-21ntpq08:00:180
687322291,4sleep0689009-21tr07:50:160
685385291,3sleep1685456-21find07:45:241
676064291,4sleep1677189-21tr07:40:151
674542291,4sleep1674605-31find07:36:301
668865292,3sleep0151ktimers/007:30:240
664332291,4sleep0664356-21munin-run07:30:000
646288291,4sleep1647411-21cut07:15:021
975303281,2sleep10-21swapper/112:25:121
969239281,3sleep0969456-21users12:15:250
968838281,3sleep2968887-21ntpq12:15:192
958716281,3sleep1954436-21/usr/sbin/munin12:05:241
958716281,3sleep1954436-21/usr/sbin/munin12:05:241
954177280,3sleep1954225-21munin-run12:05:001
954120280,1sleep20-21swapper/212:04:212
945184281,3sleep0946917-21kthreadcore11:55:170
921436281,3sleep0921481-21ntpq11:30:200
916400280,1sleep00-21swapper/011:25:240
901436280,1sleep00-21swapper/011:15:010
900277281,3sleep2900417-21/usr/sbin/munin11:10:182
891767281,3sleep1892572-21kthreadcore11:05:161
890942280,4sleep1890983-21systemd-run11:05:001
833856281,3sleep1833917-21forks10:10:131
833080280,1sleep00-21swapper/010:06:270
832411281,3sleep2832465-21ntpq10:05:342
828647280,1sleep10-21swapper/110:05:121
827428281,3sleep0827435-21sendmail_mailtr10:00:220
821873281,3sleep1821921-21ntpq09:55:191
821873281,3sleep1821921-21ntpq09:55:181
812663280,2sleep20-21swapper/209:50:022
796655280,4sleep2796673-21apt-get09:35:112
780388280,1sleep10-21swapper/109:16:101
775740281,3sleep2775883-21cstates09:15:112
771508281,3sleep0773200-21grep09:10:160
758881281,3sleep10-21swapper/108:55:231
744388281,3sleep0744483-21cpuspeed_turbos08:45:110
743129280,3sleep2743176-21ntpq08:40:202
730688281,3sleep0732400-21latency08:30:160
723392280,2sleep10-21swapper/108:25:121
706208280,3sleep2706321-21ntpq08:05:172
681645281,3sleep0682846-21grep07:45:150
653802281,2sleep0653821-21apt-get07:20:100
985338270,1sleep10-21swapper/112:32:541
985251270,1sleep20-21swapper/212:31:462
980024270,1sleep20-21swapper/212:26:542
926957271,3sleep20-21swapper/211:35:232
922041270,1sleep20-21swapper/211:30:342
921488271,3sleep10-21swapper/111:30:211
907501271,2sleep10-21swapper/111:20:151
892573270,3sleep2894297-21kthreadcore11:05:172
871279271,2sleep10-21swapper/110:45:161
863747271,3sleep2863802-21modprobe10:35:192
858794270,3sleep1858865-21find10:30:251
850208271,2sleep1851882-21kthreadcore10:25:161
847977271,2sleep00-21swapper/010:20:180
838332270,1sleep20-21swapper/210:12:182
837571270,3sleep0837618-21ntpq10:10:200
827822270,1sleep10-21swapper/110:00:511
802205271,3sleep0802419-21smartctl09:40:140
800941271,3sleep0801180-21sed09:35:250
791108270,1sleep00-21swapper/009:29:200
785954271,2sleep20-21swapper/209:25:012
754297270,2sleep00-21swapper/008:51:520
748985270,1sleep20-21swapper/208:45:332
728643270,2sleep20-21swapper/208:30:102
728269270,1sleep20-21swapper/208:28:372
692169271,3sleep00-21swapper/007:55:170
685803270,1sleep20-21swapper/207:46:332
659328271,3sleep20-21swapper/207:25:132
659306270,2sleep10-21swapper/107:25:121
646083270,4sleep20-21swapper/207:13:062
643000271,3sleep1321rcuc/107:10:171
641768271,2sleep00-21swapper/007:10:160
985764260,4sleep00-21swapper/012:35:050
974972260,1sleep00-21swapper/012:24:110
964334260,1sleep00-21swapper/012:11:440
963667261,2sleep1864791-21kworker/1:2+events12:10:211
943065260,1sleep10-21swapper/111:51:251
934860261,2sleep10-21swapper/111:45:161
928260261,2sleep00-21swapper/011:40:140
905418261,1sleep10-21swapper/111:15:181
886149261,2sleep20-21swapper/211:00:122
875517260,3sleep1321rcuc/110:50:131
869859260,4sleep00-21swapper/010:43:350
865928261,2sleep20-21swapper/210:40:162
858327260,2sleep2858434-21strings10:30:182
848797260,1sleep10-21swapper/110:22:151
839380261,2sleep20-21swapper/210:15:152
822572260,4sleep00-21swapper/009:55:370
822572260,4sleep00-21swapper/009:55:370
812674260,1sleep10-21swapper/109:50:091
811883260,1sleep10-21swapper/109:46:481
801934261,2sleep2802089-21cstates09:40:122
791067260,1sleep10-21swapper/109:28:531
775357260,4sleep20-21swapper/209:13:472
765250261,2sleep10-21swapper/109:05:121
764275261,2sleep20-21swapper/209:00:242
759552260,1sleep00-21swapper/008:57:250
737847261,1sleep00-21swapper/008:35:170
732723260,2sleep10-21swapper/108:30:211
722955260,1sleep00-21swapper/008:22:500
722843260,1sleep10-21swapper/108:21:231
717762260,1sleep20-21swapper/208:18:482
703912261,2sleep00-21swapper/008:05:150
696421260,4sleep20-21swapper/207:58:232
695616261,2sleep10-21swapper/107:55:221
690146260,2sleep20-21swapper/207:50:182
677190260,3sleep2678970-21kthreadcore07:40:162
658511261,3sleep2431ktimers/207:21:482
653264260,4sleep20-21swapper/207:16:512
979399250,2sleep00-21swapper/012:25:240
881038250,2sleep10-21swapper/110:55:141
870060250,1sleep20-21swapper/210:45:012
863852250,1sleep00-21swapper/010:35:210
848803250,3sleep20-21swapper/210:22:202
838807250,1sleep10-21swapper/110:15:101
781550250,2sleep10-21swapper/109:20:151
753451250,1sleep10-21swapper/108:50:171
712533250,3sleep20-21swapper/208:14:032
667642250,2sleep10-21swapper/107:30:171
959964240,1sleep20-21swapper/212:10:142
943874240,1sleep10-21swapper/111:55:021
770203240,1sleep10-21swapper/109:10:011
739431240,1sleep00-21swapper/008:40:140
723750240,1sleep00-21swapper/008:25:150
707597240,1sleep10-21swapper/108:10:091
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional